To alleviate the data scarcity problem in End-to-end speech translation (ST),
pre-training on data for speech recognition and machine translation is
considered as an important technique. However, the modality gap between speech
and text prevents the ST model from efficiently inheriting knowledge from the
pre-trained models. In this work, we propose AdaTranS for end-to-end ST. It
adapts the speech features with a new shrinking mechanism to mitigate the
length mismatch between speech and text features by predicting word boundaries.
Experiments on the MUST-C dataset demonstrate that AdaTranS achieves better
performance than the other shrinking-based methods, with higher inference speed
and lower memory usage. Further experiments also show that AdaTranS can be
equipped with additional alignment losses to further improve performance.

本研究提出了 AdaTranS 技术，通过预测单词边界，减少语音和文本特征之间的长度差异以适应语音特征，解决了端到端语音翻译中的数据稀缺问题。通过 MUST-C 数据集的实验表明，AdaTranS 比其他基于收缩的方法具有更好的性能，具有较高的推理速度和低内存使用率。进一步的实验还表明，AdaTranS 还可以配备额外的对齐损失来进一步提高性能。