This paper describes the submission to the IWSLT 2021 offline speech
translation task by the UPC Machine Translation group. The task consists of
building a system capable of translating English audio recordings extracted
from TED talks into German text. Submitted systems can be either cascade or
end-to-end and use a custom or given segmentation. Our submission is an
end-to-end speech translation system, which combines pre-trained models
(Wav2Vec 2.0 and mBART) with coupling modules between the encoder and decoder,
and uses an efficient fine-tuning technique, which trains only 20% of its total
parameters. We show that adding an Adapter to the system and pre-training it,
can increase the convergence speed and the final result, with which we achieve
a BLEU score of 27.3 on the MuST-C test set. Our final model is an ensemble
that obtains 28.22 BLEU score on the same set. Our submission also uses a
custom segmentation algorithm that employs pre-trained Wav2Vec 2.0 for
identifying periods of untranscribable text and can bring improvements of 2.5
to 3 BLEU score on the IWSLT 2019 test set, as compared to the result with the
given segmentation.

本文介绍了 UPC 机器翻译小组提交给 IWSLT2021 离线语音翻译任务的参赛作品，是一个基于端到端的离线语音翻译系统，采用预训练模型 Wav2Vec 2.0 和 mBART 的联合技术，在仅训练 20% 的参数的前提下获得了 27.3 的 BLEU 分数，并采用 Adapter 及预训练技术提高了收敛速度和最终结果，最后使用 Ensemble 模型取得 28.22 的 BLEU 分数，同时还使用了一种自定义细分算法，在 IWSLT2019 测试集上相对于给定细分的结果提高了 2.5 到 3 的 BLEU 分数。

使用预训练模型和适配器进行端到端的语音翻译：UPC 在 IWSLT 2021

End-to-End Speech Translation with Pre-trained Models and Adapters: UPC  at IWSLT 2021

This paper describes FBK's participation in the IWSLT 2020 offline speech
translation (ST) task. The task evaluates systems' ability to translate English
TED talks audio into German texts. The test talks are provided in two versions:
one contains the data already segmented with automatic tools and the other is
the raw data without any segmentation. Participants can decide whether to work
on custom segmentation or not. We used the provided segmentation. Our system is
an end-to-end model based on an adaptation of the Transformer for speech data.
Its training process is the main focus of this paper and it is based on: i)
transfer learning (ASR pretraining and knowledge distillation), ii) data
augmentation (SpecAugment, time stretch and synthetic data), iii) combining
synthetic and real data marked as different domains, and iv) multi-task
learning using the CTC loss. Finally, after the training with word-level
knowledge distillation is complete, our ST models are fine-tuned using label
smoothed cross entropy. Our best model scored 29 BLEU on the MuST-C En-De test
set, which is an excellent result compared to recent papers, and 23.7 BLEU on
the same data segmented with VAD, showing the need for researching solutions
addressing this specific data condition.

本文介绍了 FBK 参加 IWSLT 2020 离线语音翻译任务的方法和成果，使用基于 Transformer 模型的端到端系统，并运用了 ASR 预训练、数据增强、多任务学习等技术，最佳 BLEU 得分达到 29，优于最近相关研究。