Streaming automatic speech recognition (ASR) aims to emit each hypothesized
word as quickly and accurately as possible. However, emitting fast without
degrading quality, as measured by word error rate (WER), is highly challenging.
Existing approaches including Early and Late Penalties and Constrained
Alignments penalize emission delay by manipulating per-token or per-frame
probability prediction in sequence transducer models. While being successful in
reducing delay, these approaches suffer from significant accuracy regression
and also require additional word alignment information from an existing model.
In this work, we propose a sequence-level emission regularization method, named
FastEmit, that applies latency regularization directly on per-sequence
probability in training transducer models, and does not require any alignment.
We demonstrate that FastEmit is more suitable to the sequence-level
optimization of transducer models for streaming ASR by applying it on various
end-to-end streaming ASR networks including RNN-Transducer,
Transformer-Transducer, ConvNet-Transducer and Conformer-Transducer. We achieve
150-300 ms latency reduction with significantly better accuracy over previous
techniques on a Voice Search test set. FastEmit also improves streaming ASR
accuracy from 4.4%/8.9% to 3.1%/7.5% WER, meanwhile reduces 90th percentile
latency from 210 ms to only 30 ms on LibriSpeech.

本研究提出一种名为 FastEmit 的序列级发射规则方法，该方法可在训练转导模型时直接对序列级概率进行延迟规则化，从而在不需要任何对齐的情况下更适合流式 ASR 的序列级优化，有效降低音频识别的延迟并提升精度。