Code-switching deals with alternative languages in communication process.
Training end-to-end (E2E) automatic speech recognition (ASR) systems for
code-switching is especially challenging as code-switching training data are
always insufficient to combat the increased multilingual context confusion due
to the presence of more than one language. We propose a language-related
attention mechanism to reduce multilingual context confusion for the E2E
code-switching ASR model based on the Equivalence Constraint (EC) Theory. The
linguistic theory requires that any monolingual fragment that occurs in the
code-switching sentence must occur in one of the monolingual sentences. The
theory establishes a bridge between monolingual data and code-switching data.
We leverage this linguistics theory to design the code-switching E2E ASR model.
The proposed model efficiently transfers language knowledge from rich
monolingual data to improve the performance of the code-switching ASR model. We
evaluate our model on ASRU 2019 Mandarin-English code-switching challenge
dataset. Compared to the baseline model, our proposed model achieves a 17.12%
relative error reduction.

本文基于等价约束理论提出了一种语言相关的注意机制，将单语数据转化为多语数据应用于端到端的自动语音识别中，获得了相对误差减少 17.12% 的优异性能。