We propose and evaluate transformer-based acoustic models (AMs) for hybrid
speech recognition. Several modeling choices are discussed in this work,
including various positional embedding methods and an iterated loss to enable
training deep transformers. We also present a preliminary study of using
limited right context in transformer models, which makes it possible for
streaming applications. We demonstrate that on the widely used Librispeech
benchmark, our transformer-based AM outperforms the best published hybrid
result by 19% to 26% relative when the standard n-gram language model (LM) is
used. Combined with neural network LM for rescoring, our proposed approach
achieves state-of-the-art results on Librispeech. Our findings are also
confirmed on a much larger internal dataset.

本文提出并评估了基于 transformer 的声学模型在混合语音识别中的应用。通过讨论不同的建模选择，包括各种位置嵌入方法及迭代损失等，我们证明了在 Librispeech 基准测试中使用 transformer 模型相比其他模型性能更优越，且结合神经网络进行二次评分，我们的方法可以达到最先进的结果。