Spoken Language Understanding (SLU) is a critical component of voice
assistants; it consists of converting speech to semantic parses for task
execution. Previous works have explored end-to-end models to improve the
quality and robustness of SLU models with Deliberation, however these models
have remained autoregressive, resulting in higher latencies. In this work we
introduce PRoDeliberation, a novel method leveraging a Connectionist Temporal
Classification-based decoding strategy as well as a denoising objective to
train robust non-autoregressive deliberation models. We show that
PRoDeliberation achieves the latency reduction of parallel decoding (2-10x
improvement over autoregressive models) while retaining the ability to correct
Automatic Speech Recognition (ASR) mistranscriptions of autoregressive
deliberation systems. We further show that the design of the denoising training
allows PRoDeliberation to overcome the limitations of small ASR devices, and we
provide analysis on the necessity of each component of the system.

Spoken Language Understanding (SLU) 的关键组成部分是将语音转化为语义分析，以便进行任务执行。本研究介绍了 PRoDeliberation，一种利用基于连接主义时间分类的解码策略和降噪目标来训练稳健的非自回归思考模型的新方法。我们展示了 PRoDeliberation 实现了并行解码的延迟降低（比自回归模型提高 2-10 倍），同时保持了修正自回归思考系统的自动语音识别（ASR）误识别的能力。此外，我们进一步展示了降噪训练的设计使得 PRoDeliberation 能够克服小型 ASR 设备的限制，并对系统的每个组件的必要性进行了分析。

PRoDeliberation：面向端到端口语理解的并行强化沟通

PRoDeliberation: Parallel Robust Deliberation for End-to-End Spoken  Language Understanding

Contrastive-learning-based methods have dominated sentence representation
learning. These methods regularize the representation space by pulling similar
sentence representations closer and pushing away the dissimilar ones and have
been proven effective in various NLP tasks, e.g., semantic textual similarity
(STS) tasks. However, it is challenging for these methods to learn fine-grained
semantics as they only learn from the inter-sentence perspective, i.e., their
supervision signal comes from the relationship between data samples. In this
work, we propose a novel denoising objective that inherits from another
perspective, i.e., the intra-sentence perspective. By introducing both discrete
and continuous noise, we generate noisy sentences and then train our model to
restore them to their original form. Our empirical evaluations demonstrate that
this approach delivers competitive results on both semantic textual similarity
(STS) and a wide range of transfer tasks, standing up well in comparison to
contrastive-learning-based methods. Notably, the proposed intra-sentence
denoising objective complements existing inter-sentence contrastive
methodologies and can be integrated with them to further enhance performance.
Our code is available at this https URL

通过引入离散和连续噪声，我们提出了一种新的去噪目标，通过训练模型将噪声句子恢复为其原始形式，从而在语义文本相似性和广泛的迁移任务中实现了有竞争力的结果。

DenoSent: 自监督句子表示学习的去噪目标

DenoSent: A Denoising Objective for Self-Supervised Sentence  Representation Learning

Despite the success of multilingual sequence-to-sequence pretraining, most
existing approaches rely on monolingual corpora, and do not make use of the
strong cross-lingual signal contained in parallel data. In this paper, we
present PARADISE (PARAllel & Denoising Integration in SEquence-to-sequence
models), which extends the conventional denoising objective used to train these
models by (i) replacing words in the noised sequence according to a
multilingual dictionary, and (ii) predicting the reference translation
according to a parallel corpus instead of recovering the original sequence. Our
experiments on machine translation and cross-lingual natural language inference
show an average improvement of 2.0 BLEU points and 6.7 accuracy points from
integrating parallel data into pretraining, respectively, obtaining results
that are competitive with several popular models at a fraction of their
computational cost.

该研究介绍了 PARADISE 方法，通过将多语言字典和平行语料库用于训练模型的噪声序列中，取得了与其他模型相当、计算成本更低的 2.0 BLEU 点平均提升和 6.7 准确度点的跨语言自然语言推理和机器翻译实验结果。