In this work, we investigate the effect of language models (LMs) with different context lengths and label units (phoneme vs. word) used in sequence discriminative training for phoneme-based neural transducers. Both lattice-free and N-best-list approaches are examined. For lattice-free methods with phoneme-level LMs, we propose a method to approximate the context history to employ LMs with full-context dependency. This approximation can be extended to arbitrary context length and enables the usage of word-level LMs in lattice-free methods. Moreover, a systematic comparison is conducted across lattice-free and N-best-list-based methods. Experimental results on Librispeech show that using the word-level LM in training outperforms the phoneme-level LM. Besides, we find that the context size of the LM used for probability computation has a limited effect on performance. Moreover, our results reveal the pivotal importance of the hypothesis space quality in sequence discriminative training.

该研究探讨了不同上下文长度和标签单位（音素与词）在基于音素的神经传递器的序列鉴别式训练中，语言模型（LMs）的影响。研究结果表明，使用词级别的LM在训练中优于音素级别的LM，并且概率计算所使用的LM的上下文大小对性能有限影响，同时还揭示了序列鉴别式训练中假设空间质量的重要性。

探究语言模型在序列判别式训练神经转录器中的影响