In speech recognition, it is essential to model the phonetic content of the
input signal while discarding irrelevant factors such as speaker variations and
noise, which is challenging in low-resource settings. Self-supervised
pre-training has been proposed as a way to improve both supervised and
unsupervised speech recognition, including frame-level feature representations
and Acoustic Word Embeddings (AWE) for variable-length segments. However,
self-supervised models alone cannot learn perfect separation of the linguistic
content as they are trained to optimize indirect objectives. In this work, we
experiment with different pre-trained self-supervised features as input to AWE
models and show that they work best within a supervised framework. Models
trained on English can be transferred to other languages with no adaptation and
outperform self-supervised models trained solely on the target languages.

本文研究了使用自监督预训练技术来提高语音识别的准确性，发现在监督学习框架中，利用不同的预训练自监督特征作为 Acoustic Word Embeddings 的输入是最有效的方法，并且这种方法可实现跨语言迁移。

有监督的声学嵌入及其在不同语言之间的可转移性

Supervised Acoustic Embeddings And Their Transferability Across Languages

In this paper, we explore vector quantization for acoustic unit discovery.
Leveraging unlabelled data, we aim to learn discrete representations of speech
that separate phonetic content from speaker-specific details. We propose two
neural models to tackle this challenge - both use vector quantization to map
continuous features to a finite set of codes. The first model is a type of
vector-quantized variational autoencoder (VQ-VAE). The VQ-VAE encodes speech
into a sequence of discrete units before reconstructing the audio waveform. Our
second model combines vector quantization with contrastive predictive coding
(VQ-CPC). The idea is to learn a representation of speech by predicting future
acoustic units. We evaluate the models on English and Indonesian data for the
ZeroSpeech 2020 challenge. In ABX phone discrimination tests, both models
outperform all submissions to the 2019 and 2020 challenges, with a relative
improvement of more than 30%. The models also perform competitively on a
downstream voice conversion task. Of the two, VQ-CPC performs slightly better
in general and is simpler and faster to train. Finally, probing experiments
show that vector quantization is an effective bottleneck, forcing the models to
discard speaker information.

本文提出两种神经模型，均使用向量量化技术将连续特征映射为有限的编码，旨在利用无标签数据学习将语音的音素内容与说话人特定细节分离的离散表述。在 Zero Speech 2020 挑战赛的英语和印度尼西亚语数据上，我们的两个模型都优于 2019 年和 2020 年挑战赛的所有提交，相对提高超过 30%。

零语音 2020 挑战中基于向量量化的神经网络用于语音单元发现

Vector-quantized neural networks for acoustic unit discovery in the  ZeroSpeech 2020 challenge

Training objectives based on predictive coding have recently been shown to be
very effective at learning meaningful representations from unlabeled speech.
One example is Autoregressive Predictive Coding (Chung et al., 2019), which
trains an autoregressive RNN to generate an unseen future frame given a context
such as recent past frames. The basic hypothesis of these approaches is that
hidden states that can accurately predict future frames are a useful
representation for many downstream tasks. In this paper we extend this
hypothesis and aim to enrich the information encoded in the hidden states by
training the model to make more accurate future predictions. We propose an
auxiliary objective that serves as a regularization to improve generalization
of the future frame prediction task. Experimental results on phonetic
classification, speech recognition, and speech translation not only support the
hypothesis, but also demonstrate the effectiveness of our approach in learning
representations that contain richer phonetic content.

通过未标记的语音学习有意义的表示方法，将预测编码的培训目标应用于自回归预测编码（ARPC），并提出了一种辅助目标以提高未来帧预测任务的泛化能力和语音内容。不仅支持假设，还证明了该方法在学习含有更丰富语音内容的表示方面的有效性。