Utilizing text-only data with an external language model (ELM) in end-to-end
RNN-Transducer (RNN-T) for speech recognition is challenging. Recently, a class
of methods such as density ratio (DR) and internal language model estimation
(ILME) have been developed, outperforming the classic shallow fusion (SF)
method. The basic idea behind these methods is that RNN-T posterior should
first subtract the implicitly learned internal language model (ILM) prior, in
order to integrate the ELM. While recent studies suggest that RNN-T only learns
some low-order language model information, the DR method uses a well-trained
neural language model with full context, which may be inappropriate for the
estimation of ILM and deteriorate the integration performance. Based on the DR
method, we propose a low-order density ratio method (LODR) by replacing the
estimation with a low-order weak language model. Extensive empirical
experiments are conducted on both in-domain and cross-domain scenarios on
English LibriSpeech & Tedlium-2 and Chinese WenetSpeech & AISHELL-1 datasets.
It is shown that LODR consistently outperforms SF in all tasks, while
performing generally close to ILME and better than DR in most tests.

在端到端的 RNN-Transducer（RNN-T）中使用仅文本数据是具有挑战性的，本文研究了使用密度比和内部语言模型（ILM）估计的方法来将外部语言模型（ELM）集成到 RNN-T 中以实现语音识别，并提出了一种低阶密度比方法（LODR），在 LibriSpeech，Tedlium-2，WenetSpeech 和 AISHELL-1 数据集上的大量实验表明，LODR 始终优于 SF，在大多数测试中表现优于 DR 且一般性能接近 ILME。