In this paper, we propose a language-universal adapter learning framework
based on a pre-trained model for end-to-end multilingual automatic speech
recognition (ASR). For acoustic modeling, the wav2vec 2.0 pre-trained model is
fine-tuned by inserting language-specific and language-universal adapters. An
online knowledge distillation is then used to enable the language-universal
adapters to learn both language-specific and universal features. The linguistic
information confusion is also reduced by leveraging language identifiers
(LIDs). With LIDs we perform a position-wise modification on the multi-head
attention outputs. In the inference procedure, the language-specific adapters
are removed while the language-universal adapters are kept activated. The
proposed method improves the recognition accuracy and addresses the linear
increase of the number of adapters' parameters with the number of languages in
common multilingual ASR systems. Experiments on the BABEL dataset confirm the
effectiveness of the proposed framework. Compared to the conventional
multilingual model, a 3.3% absolute error rate reduction is achieved. The code
is available at: this https URL.

本文提出了一个基于预训练模型的语言通用适配器学习框架，用于端到端的多语言自动语音识别。通过插入特定于语言和语言通用适配器来微调 Wav2Vec 2.0 预训练模型，然后使用在线知识蒸馏来使语言通用适配器学习特定于语言和通用特征，通过利用语言识别器（LID）降低语言信息混淆，进而改善识别准确率，并解决通用多语种 ASR 系统中适配器参数数量随语言数量线性增长的问题，BABEL 数据集上的实验结果验证了所提出框架的有效性，并相比传统的多语种模型取得了 3.3％的绝对误差率的降低。