Neural transducers have been widely used in automatic speech recognition (ASR). In this paper, we introduce it to streaming end-to-end speech translation (ST), which aims to convert audio signals to texts in other languages directly. Compared with cascaded ST that performs ASR followed by text-based machine translation (MT), the proposed Transformer transducer (TT)-based ST model drastically reduces inference latency, exploits speech information, and avoids error propagation from ASR to MT. To improve the modeling capacity, we propose attention pooling for the joint network in TT. In addition, we extend TT-based ST to multilingual ST, which generates texts of multiple languages at the same time. Experimental results on a large-scale 50 thousand (K) hours pseudo-labeled training set show that TT-based ST not only significantly reduces inference time but also outperforms non-streaming cascaded ST for English-German translation.

本文介绍了如何将神经转录器引入流式端到端语音翻译（ST）中，提出了基于注意力池化的Transformer transducer（TT）模型以及在多语言ST中的应用，结果表明TT模型不仅显著减少了推理时间，而且在英德翻译上优于基于ASR和MT的非流式级联ST。

大规模流式端到端语音翻译基于神经转录器