Neural transducers have been widely used in automatic speech recognition
(ASR). In this paper, we introduce it to streaming end-to-end speech
translation (ST), which aims to convert audio signals to texts in other
languages directly. Compared with cascaded ST that performs ASR followed by
text-based machine translation (MT), the proposed Transformer transducer
(TT)-based ST model drastically reduces inference latency, exploits speech
information, and avoids error propagation from ASR to MT. To improve the
modeling capacity, we propose attention pooling for the joint network in TT. In
addition, we extend TT-based ST to multilingual ST, which generates texts of
multiple languages at the same time. Experimental results on a large-scale 50
thousand (K) hours pseudo-labeled training set show that TT-based ST not only
significantly reduces inference time but also outperforms non-streaming
cascaded ST for English-German translation.

本文介绍了如何将神经转录器引入流式端到端语音翻译（ST）中，提出了基于注意力池化的 Transformer transducer（TT）模型以及在多语言 ST 中的应用，结果表明 TT 模型不仅显著减少了推理时间，而且在英德翻译上优于基于 ASR 和 MT 的非流式级联 ST。