Sequence-to-sequence models have been widely used in end-to-end speech processing, for example, automatic speech recognition (ASR), speech translation (ST), and text-to-speech (TTS). This paper focuses on an emergent sequence-to-sequence model called Transformer, which achieves state-of-the-art performance in neural machine translation and other natural language processing applications. We undertook intensive studies in which we experimentally compared and analyzed Transformer and conventional recurrent neural networks (RNN) in a total of 15 ASR, one multilingual ASR, one ST, and two TTS benchmarks. Our experiments revealed various training tips and significant performance benefits obtained with Transformer for each task including the surprising superiority of Transformer in 13/15 ASR benchmarks in comparison with RNN. We are preparing to release Kaldi-style reproducible recipes using open source and publicly available datasets for all the ASR, ST, and TTS tasks for the community to succeed our exciting outcomes.

本文章研究了一种新兴的序列到序列模型——Transformer，并将它与传统RNN模型在15项ASR、1项多语言ASR、1项ST和2项TTS基准测试中进行了比较和分析，发现了各种训练技巧以及Transformer在各项任务中带来的显著性能优势，最终实验结果表明Transformer在13/15项ASR基准测试中比RNN具有惊人的优越性能。

基于Transformer与RNN的语音应用比较研究