The bi-encoder structure has been intensively investigated in code-switching
(CS) automatic speech recognition (ASR). However, most existing methods require
the structures of two monolingual ASR models (MAMs) should be the same and only
use the encoder of MAMs. This leads to the problem that pre-trained MAMs cannot
be timely and fully used for CS ASR. In this paper, we propose a monolingual
recognizers fusion method for CS ASR. It has two stages: the speech awareness
(SA) stage and the language fusion (LF) stage. In the SA stage, acoustic
features are mapped to two language-specific predictions by two independent
MAMs. To keep the MAMs focused on their own language, we further extend the
language-aware training strategy for the MAMs. In the LF stage, the BELM fuses
two language-specific predictions to get the final prediction. Moreover, we
propose a text simulation strategy to simplify the training process of the BELM
and reduce reliance on CS data. Experiments on a Mandarin-English corpus show
the efficiency of the proposed method. The mix error rate is significantly
reduced on the test set after using open-source pre-trained MAMs.

本文提出了一种单语言识别器融合方法用于语码混合自动语音识别，通过两个阶段实现：语音感知阶段和语言融合阶段，并提出了文本模拟策略简化 BELM 训练过程并减少对语码混合数据的依赖。实验表明，使用开源预训练的 MAM 后，测试集上的混合错误率显著降低。