Recently, large pre-trained multilingual speech models have shown potential
in scaling Automatic Speech Recognition (ASR) to many low-resource languages.
Some of these models employ language adapters in their formulation, which helps
to improve monolingual performance and avoids some of the drawbacks of
multi-lingual modeling on resource-rich languages. However, this formulation
restricts the usability of these models on code-switched speech, where two
languages are mixed together in the same utterance. In this work, we propose
ways to effectively fine-tune such models on code-switched speech, by
assimilating information from both language adapters at each language
adaptation point in the network. We also model code-switching as a sequence of
latent binary sequences that can be used to guide the flow of information from
each language adapter at the frame level. The proposed approaches are evaluated
on three code-switched datasets encompassing Arabic, Mandarin, and Hindi
languages paired with English, showing consistent improvements in
code-switching performance with at least 10\% absolute reduction in CER across
all test sets.

本文提出了在代码转换的语音中有效地微调大型预训练多语言语音模型的方法，并将代码转换建模为一系列潜在的二进制序列，以引导每个语言适配器在帧级别上的信息流，通过在阿拉伯语、普通话和印地语等多种语言与英语配对的数据集上的评估，显示出代码转换性能的持续改进，至少可以将语音识别错误率 (CER) 降低 10％。