We focus on the problem of language modeling for code-switched language, in
the context of automatic speech recognition (ASR). Language modeling for
code-switched language is challenging for (at least) three reasons: (1) lack of
available large-scale code-switched data for training; (2) lack of a replicable
evaluation setup that is ASR directed yet isolates language modeling
performance from the other intricacies of the ASR system; and (3) the reliance
on generative modeling. We tackle these three issues: we propose an
ASR-motivated evaluation setup which is decoupled from an ASR system and the
choice of vocabulary, and provide an evaluation dataset for English-Spanish
code-switching. This setup lends itself to a discriminative training approach,
which we demonstrate to work better than generative language modeling. Finally,
we explore a variety of training protocols and verify the effectiveness of
training with large amounts of monolingual data followed by fine-tuning with
small amounts of code-switched data, for both the generative and discriminative
cases.

本文研究语言建模在代表代码切换语言的语境下在自动语音识别（ASR）中所存在的难题，针对三个难点分别提出解决方案：如何从缺乏大规模训练数据、ASR 系统中与语言建模相关的性能测试的实现方法以及基于生成式模型所存在的缺陷，我们构建了一个独立于 ASR 系统和词汇选择的评估数据集和评估方式，并采用一种判别式训练方法，证明其比生成式更加有效。最后，我们探索了多种训练协议，并验证了使用大量单语数据随后加上小量的代码切换数据来进行精调训练的可行性。