In this work, we investigate the effect of language models (LMs) with
different context lengths and label units (phoneme vs. word) used in sequence
discriminative training for phoneme-based neural transducers. Both lattice-free
and N-best-list approaches are examined. For lattice-free methods with
phoneme-level LMs, we propose a method to approximate the context history to
employ LMs with full-context dependency. This approximation can be extended to
arbitrary context length and enables the usage of word-level LMs in
lattice-free methods. Moreover, a systematic comparison is conducted across
lattice-free and N-best-list-based methods. Experimental results on Librispeech
show that using the word-level LM in training outperforms the phoneme-level LM.
Besides, we find that the context size of the LM used for probability
computation has a limited effect on performance. Moreover, our results reveal
the pivotal importance of the hypothesis space quality in sequence
discriminative training.

该研究探讨了不同上下文长度和标签单位（音素与词）在基于音素的神经传递器的序列鉴别式训练中，语言模型（LMs）的影响。研究结果表明，使用词级别的 LM 在训练中优于音素级别的 LM，并且概率计算所使用的 LM 的上下文大小对性能有限影响，同时还揭示了序列鉴别式训练中假设空间质量的重要性。

探究语言模型在序列判别式训练神经转录器中的影响

Investigating the Effect of Language Models in Sequence Discriminative  Training for Neural Transducers

Internal language model (ILM) subtraction has been widely applied to improve
the performance of the RNN-Transducer with external language model (LM) fusion
for speech recognition. In this work, we show that sequence discriminative
training has a strong correlation with ILM subtraction from both theoretical
and empirical points of view. Theoretically, we derive that the global optimum
of maximum mutual information (MMI) training shares a similar formula as ILM
subtraction. Empirically, we show that ILM subtraction and sequence
discriminative training achieve similar performance across a wide range of
experiments on Librispeech, including both MMI and minimum Bayes risk (MBR)
criteria, as well as neural transducers and LMs of both full and limited
context. The benefit of ILM subtraction also becomes much smaller after
sequence discriminative training. We also provide an in-depth study to show
that sequence discriminative training has a minimal effect on the commonly used
zero-encoder ILM estimation, but a joint effect on both encoder and prediction
+ joint network for posterior probability reshaping including both ILM and
blank suppression.

内部语言模型减法与序列鉴别训练在语音识别中表现类似，通过实证研究证明二者性能相近，同时了解序列鉴别训练对常用零编码器内部语言模型估计产生极小影响，但对包括内部语言模型和空白抑制的编码器和预测 + 联合网络的后验概率重整具有联合效应。

关于内部语言模型与序列判别式训练在神经传达器中的关系

On the Relation between Internal Language Model and Sequence  Discriminative Training for Neural Transducers

Speech recognition is a sequence prediction problem. Besides employing
various deep learning approaches for framelevel classification, sequence-level
discriminative training has been proved to be indispensable to achieve the
state-of-the-art performance in large vocabulary continuous speech recognition
(LVCSR). However, keyword spotting (KWS), as one of the most common speech
recognition tasks, almost only benefits from frame-level deep learning due to
the difficulty of getting competing sequence hypotheses. The few studies on
sequence discriminative training for KWS are limited for fixed vocabulary or
LVCSR based methods and have not been compared to the state-of-the-art deep
learning based KWS approaches. In this paper, a sequence discriminative
training framework is proposed for both fixed vocabulary and unrestricted
acoustic KWS. Sequence discriminative training for both sequence-level
generative and discriminative models are systematically investigated. By
introducing word-independent phone lattices or non-keyword blank symbols to
construct competing hypotheses, feasible and efficient sequence discriminative
training approaches are proposed for acoustic KWS. Experiments showed that the
proposed approaches obtained consistent and significant improvement in both
fixed vocabulary and unrestricted KWS tasks, compared to previous frame-level
deep learning based acoustic KWS methods.

本文探讨了基于深度学习框架下的固定词汇量和无限制口语识别中，序列鉴别式训练的有效性，提出了一种有效的方法，通过引入独立音素网格或非关键词空白符号来构建竞争性假设，相比前面基于深度学习的原始方法，在固定词汇和无限制 KWS 任务中，提高了很大的识别性能。