Sequence transducers, such as the RNN-T and the Conformer-T, are one of the
most promising models of end-to-end speech recognition, especially in streaming
scenarios where both latency and accuracy are important. Although various
methods, such as alignment-restricted training and FastEmit, have been studied
to reduce the latency, latency reduction is often accompanied with a
significant degradation in accuracy. We argue that this suboptimal performance
might be caused because none of the prior methods explicitly model and reduce
the latency. In this paper, we propose a new training method to explicitly
model and reduce the latency of sequence transducer models. First, we define
the expected latency at each diagonal line on the lattice, and show that its
gradient can be computed efficiently within the forward-backward algorithm.
Then we augment the transducer loss with this expected latency, so that an
optimal trade-off between latency and accuracy is achieved. Experimental
results on the WSJ dataset show that the proposed minimum latency training
reduces the latency of causal Conformer-T from 220 ms to 27 ms within a WER
degradation of 0.7%, and outperforms conventional alignment-restricted training
(110 ms) and FastEmit (67 ms) methods.

本文提出了一种新的训练方法，通过明确建模和减少序列变换器模型的延迟来达到最优的延迟和准确性均衡，实验结果表明，所提出的最小延迟训练方法将有助于将哈尔滨工业大学卡苏耶延迟从 220ms 降低到 27ms，并在误识率降低了 0.7% 的同时，优于传统的对齐限制训练和快速发射方法。