Recent works in end-to-end speech-to-text translation (ST) have proposed
multi-tasking methods with soft parameter sharing which leverage machine
translation (MT) data via secondary encoders that map text inputs to an
eventual cross-modal representation. In this work, we instead propose a ST/MT
multi-tasking framework with hard parameter sharing in which all model
parameters are shared cross-modally. Our method reduces the speech-text
modality gap via a pre-processing stage which converts speech and text inputs
into two discrete token sequences of similar length -- this allows models to
indiscriminately process both modalities simply using a joint vocabulary. With
experiments on MuST-C, we demonstrate that our multi-tasking framework improves
attentional encoder-decoder, Connectionist Temporal Classification (CTC),
transducer, and joint CTC/attention models by an average of +0.5 BLEU without
any external MT data. Further, we show that this framework incorporates
external MT data, yielding +0.8 BLEU, and also improves transfer learning from
pre-trained textual models, yielding +1.8 BLEU.

最近的研究在端到端语音到文本翻译（ST）中提出了多任务方法，通过辅助编码器将机器翻译（MT）数据映射到最终的跨模态表示中，采用软参数共享。本研究提出了一种具有硬参数共享的 ST/MT 多任务框架，通过预处理阶段将语音和文本输入转换为两个长度相似的离散令牌序列，从而减少了语音 - 文本模态差异。通过在 MuST-C 上的实验，我们证明了我们的多任务框架平均提高了 0.5 BLEU 的关注编码器 - 解码器、连接时序分类（CTC）、转录器和联合 CTC / 关注模型，而无需外部 MT 数据。此外，我们还展示了这个框架的融入外部 MT 数据可以产生 0.8 BLEU 的改进，并且可以提高从预训练的文本模型进行迁移学习，从而产生 1.8 BLEU 的改进。