Dual-encoder structure successfully utilizes two language-specific encoders
(LSEs) for code-switching speech recognition. Because LSEs are initialized by
two pre-trained language-specific models (LSMs), the dual-encoder structure can
exploit sufficient monolingual data and capture the individual language
attributes. However, most existing methods have no language constraints on LSEs
and underutilize language-specific knowledge of LSMs. In this paper, we propose
a language-specific characteristic assistance (LSCA) method to mitigate the
above problems. Specifically, during training, we introduce two
language-specific losses as language constraints and generate corresponding
language-specific targets for them. During decoding, we take the decoding
abilities of LSMs into account by combining the output probabilities of two
LSMs and the mixture model to obtain the final predictions. Experiments show
that either the training or decoding method of LSCA can improve the model's
performance. Furthermore, the best result can obtain up to 15.4% relative error
reduction on the code-switching test set by combining the training and decoding
methods of LSCA. Moreover, the system can process code-switching speech
recognition tasks well without extra shared parameters or even retraining based
on two pre-trained LSMs by using our method.

本文提出了一种语言特异性辅助方法（LSCA），用于改善现有方法中 LSEs 的语言约束问题。通过在训练期间引入两个语言特定的损失以及对应的语言特定目标，同时考虑 LSMs 的解码能力，LSCA 方法可以提高模型的性能，最好的结果可以相对误差降低 15.4%。 此外，该系统不需要额外的共享参数或根据两个预训练 LSMs 进行重新训练，可以很好地处理语言切换的语音识别任务。