In recent years, there has been an increasing focus on user convenience,
leading to increased interest in text-based keyword enrollment systems for
keyword spotting (KWS). Since the system utilizes text input during the
enrollment phase and audio input during actual usage, we call this task
audio-text based KWS. To enable this task, both acoustic and text encoders are
typically trained using deep metric learning loss functions, such as triplet-
and proxy-based losses. This study aims to improve existing methods by
leveraging the structural relations within acoustic embeddings and within text
embeddings. Unlike previous studies that only compare acoustic and text
embeddings on a point-to-point basis, our approach focuses on the relational
structures within the embedding space by introducing the concept of Relational
Proxy Loss (RPL). By incorporating RPL, we demonstrated improved performance on
the Wall Street Journal (WSJ) corpus.

通过引入关系型代理损失（RPL）的概念，利用声学嵌入和文本嵌入之间的关系结构，改进了现有方法，在华尔街日报（WSJ）语料库上展示了更好的性能。

基于音频文本的关键词检测的关系代理损失

Relational Proxy Loss for Audio-Text based Keyword Spotting

Recently, end-to-end models have been widely used in automatic speech
recognition (ASR) systems. Two of the most representative approaches are
connectionist temporal classification (CTC) and attention-based encoder-decoder
(AED) models. Autoregressive transformers, variants of AED, adopt an
autoregressive mechanism for token generation and thus are relatively slow
during inference. In this paper, we present a comprehensive study of a CTC
Alignment-based Single-Step Non-Autoregressive Transformer (CASS-NAT) for
end-to-end ASR. In CASS-NAT, word embeddings in the autoregressive transformer
(AT) are substituted with token-level acoustic embeddings (TAE) that are
extracted from encoder outputs with the acoustical boundary information offered
by the CTC alignment. TAE can be obtained in parallel, resulting in a parallel
generation of output tokens. During training, Viterbi-alignment is used for TAE
generation, and multiple training strategies are further explored to improve
the word error rate (WER) performance. During inference, an error-based
alignment sampling method is investigated in depth to reduce the alignment
mismatch in the training and testing processes. Experimental results show that
the CASS-NAT has a WER that is close to AT on various ASR tasks, while
providing a ~24x inference speedup. With and without self-supervised learning,
we achieve new state-of-the-art results for non-autoregressive models on
several datasets. We also analyze the behavior of the CASS-NAT decoder to
explain why it can perform similarly to AT. We find that TAEs have similar
functionality to word embeddings for grammatical structures, which might
indicate the possibility of learning some semantic information from TAEs
without a language model.

本论文介绍了一种 CTC Alignment-based Single-Step Non-Autoregressive Transformer（CASS-NAT）方法，用于自动语音识别，通过利用与 CTC 对齐的语音边界信息提取标记级别音频嵌入来提高推理速度，实现了自监督学习，提出了多个训练策略来改善单词错误率（WER）表现，并探究了基于误差的对齐采样方法以减少训练和测试过程中的对齐不匹配，实验结果表明 CASS-NAT 对于多个 ASR 任务具有接近于 AT 的 WER，同时提供了～24 倍的推理加速，并且未经过语言模型的情况下，实现了新的最高效果。

一种基于 CTC 对齐的非自回归变压器模型用于端到端自动语音识别

A CTC Alignment-based Non-autoregressive Transformer for End-to-end  Automatic Speech Recognition

Variation in speech is often quantified by comparing phonetic transcriptions
of the same utterance. However, manually transcribing speech is time-consuming
and error prone. As an alternative, therefore, we investigate the extraction of
acoustic embeddings from several self-supervised neural models. We use these
representations to compute word-based pronunciation differences between
non-native and native speakers of English, and between Norwegian dialect
speakers. For comparison with several earlier studies, we evaluate how well
these differences match human perception by comparing them with available human
judgements of similarity. We show that speech representations extracted from a
specific type of neural model (i.e. Transformers) lead to a better match with
human perception than two earlier approaches on the basis of phonetic
transcriptions and MFCC-based acoustic features. We furthermore find that
features from the neural models can generally best be extracted from one of the
middle hidden layers than from the final layer. We also demonstrate that neural
speech representations not only capture segmental differences, but also
intonational and durational differences that cannot adequately be represented
by a set of discrete symbols used in phonetic transcriptions.

研究使用自我监督的神经模型从语音中提取声学嵌入，计算非英语母语和英语母语以及挪威方言发音之间的基于单词的差异，并发现使用转换器的神经模型提取的语音表示与人类感知匹配得更好。