Self-supervised learning (SSL) is a powerful tool that allows learning of underlying representations from unlabeled data. Transformer based models such as wav2vec 2.0 and HuBERT are leading the field in the speech domain. Generally these models are fine-tuned on a small amount of labeled data for a downstream task such as Automatic Speech Recognition (ASR). This involves re-training the majority of the model for each task. Adapters are small lightweight modules which are commonly used in Natural Language Processing (NLP) to adapt pre-trained models to new tasks. In this paper we propose applying adapters to wav2vec 2.0 to reduce the number of parameters required for downstream ASR tasks, and increase scalability of the model to multiple tasks or languages. Using adapters we can perform ASR while training fewer than 10% of parameters per task compared to full fine-tuning with little degradation of performance. Ablations show that applying adapters into just the top few layers of the pre-trained network gives similar performance to full transfer, supporting the theory that higher pre-trained layers encode more phonemic information, and further optimizing efficiency.

本文研究使用适配器将预训练的wav2vec 2.0模型进行微调，以减少自动语音识别任务对参数的需求，提高模型的可扩展性和效率。应用适配器后，每个任务的参数少于10％，性能降低很少。结果表明，通过仅向预训练网络的前几层添加适配器，就可以实现与完全迁移相似的性能，进一步优化了效率。

自监督语音模型的有效适配器传递用于自动语音识别