It is challenging to train and deploy Transformer LMs for hybrid speech
recognition 2nd pass re-ranking in low-resource languages due to (1) data
scarcity in low-resource languages, (2) expensive computing costs for training
and refreshing 100+ monolingual models, and (3) hosting inefficiency
considering sparse traffic. In this study, we present a new way to group
multiple low-resource locales together and optimize the performance of
Multilingual Transformer LMs in ASR. Our Locale-group Multilingual Transformer
LMs outperform traditional multilingual LMs along with reducing maintenance
costs and operating expenses. Further, for low-resource but high-traffic
locales where deploying monolingual models is feasible, we show that
fine-tuning our locale-group multilingual LMs produces better monolingual LM
candidates than baseline monolingual LMs.

通过将多个低资源语言聚集在一起来实现地区组多语言 Transformer LM 的最佳性能，使得 Monolingual LM 候选模型表现更佳且减少维护成本和运营费用。

面向低资源语言的多语种 Transformer 语言模型

Multilingual Transformer Language Model for Speech Recognition in Low-resource Languages

We propose and evaluate transformer-based acoustic models (AMs) for hybrid
speech recognition. Several modeling choices are discussed in this work,
including various positional embedding methods and an iterated loss to enable
training deep transformers. We also present a preliminary study of using
limited right context in transformer models, which makes it possible for
streaming applications. We demonstrate that on the widely used Librispeech
benchmark, our transformer-based AM outperforms the best published hybrid
result by 19% to 26% relative when the standard n-gram language model (LM) is
used. Combined with neural network LM for rescoring, our proposed approach
achieves state-of-the-art results on Librispeech. Our findings are also
confirmed on a much larger internal dataset.

本文提出并评估了基于 transformer 的声学模型在混合语音识别中的应用。通过讨论不同的建模选择，包括各种位置嵌入方法及迭代损失等，我们证明了在 Librispeech 基准测试中使用 transformer 模型相比其他模型性能更优越，且结合神经网络进行二次评分，我们的方法可以达到最先进的结果。