RNN-Transducer (RNN-T) models have become synonymous with streaming
end-to-end ASR systems. While they perform competitively on a number of
evaluation categories, rare words pose a serious challenge to RNN-T models. One
main reason for the degradation in performance on rare words is that the
language model (LM) internal to RNN-Ts can become overconfident and lead to
hallucinated predictions that are acoustically inconsistent with the underlying
speech. To address this issue, we propose a lightweight adaptive LM discounting
technique AdaptLMD, that can be used with any RNN-T architecture without
requiring any external resources or additional parameters. AdaptLMD uses a
two-pronged approach: 1) Randomly mask the prediction network output to
encourage the RNN-T to not be overly reliant on it's outputs. 2) Dynamically
choose when to discount the implicit LM (ILM) based on rarity of recently
predicted tokens and divergence between ILM and implicit acoustic model (IAM)
scores. Comparing AdaptLMD to a competitive RNN-T baseline, we obtain up to 4%
and 14% relative reductions in overall WER and rare word PER, respectively, on
a conversational, code-mixed Hindi-English ASR task.

本文针对 RNN-T 在稀有词识别中的性能下降问题，提出了一种轻量级自适应 LM 折扣技术 AdaptLMD，通过在内部语言模型中动态选择性折扣，随机掩盖预测网络输出等方法，比较基准模型，相对于会话式的代码混合 Hindi - 英语 ASR 任务，在整体 WER 和稀有词 PER 方面分别获得了 4％和 14％的相对性能提升。