Recent success in speech representation learning enables a new way to
leverage unlabeled data to train speech recognition model. In speech
representation learning, a large amount of unlabeled data is used in a
self-supervised manner to learn a feature representation. Then a smaller amount
of labeled data is used to train a downstream ASR system using the new feature
representations. Based on our previous work DeCoAR and inspirations from other
speech representation learning, we propose DeCoAR 2.0, a Deep Contextualized
Acoustic Representation with vector quantization. We introduce several
modifications over the DeCoAR: first, we use Transformers in encoding module
instead of LSTMs; second, we introduce a vector quantization layer between
encoder and reconstruction modules; third, we propose an objective that
combines the reconstructive loss with vector quantization diversity loss to
train speech representations. Our experiments show consistent improvements over
other speech representations in different data-sparse scenarios. Without
fine-tuning, a light-weight ASR model trained on 10 hours of LibriSpeech
labeled data with DeCoAR 2.0 features outperforms the model trained on the full
960-hour dataset with filterbank features.

文章介绍了使用语音表征学习的方法在没有标签的数据上训练语音识别模型的新方法 DeCoAR 2.0。该模型采用 Transformers 编码模型，引入了向量量化层来对语音表征进行训练，并在多个数据稀疏的场景下表现出一致的提高。