An inferior performance of the streaming automatic speech recognition models
versus non-streaming model is frequently seen due to the absence of future
context. In order to improve the performance of the streaming model and reduce
the computational complexity, a frame-level model using efficient augment
memory transformer block and dynamic latency training method is employed for
streaming automatic speech recognition in this paper. The long-range history
context is stored into the augment memory bank as a complement to the limited
history context used in the encoder. Key and value are cached by a cache
mechanism and reused for next chunk to reduce computation. Afterwards, a
dynamic latency training method is proposed to obtain better performance and
support low and high latency inference simultaneously. Our experiments are
conducted on benchmark 960h LibriSpeech data set. With an average latency of
640ms, our model achieves a relative WER reduction of 6.0% on test-clean and
3.0% on test-other versus the truncate chunk-wise Transformer.

本文提出一种帧级模型，采用高效增强记忆变压器块和动态延迟训练方法来提高流媒体自动语音识别的性能和降低计算复杂性，通过缓存机制并重复使用结合增强记忆库作为有限历史上下文的补充，以达到更好的性能。实验表明，该模型在 LibriSpeech 数据集上具有较高的性能。