Adapter modules were recently introduced as an efficient alternative to
fine-tuning in NLP. Adapter tuning consists in freezing pretrained parameters
of a model and injecting lightweight modules between layers, resulting in the
addition of only a small number of task-specific trainable parameters. While
adapter tuning was investigated for multilingual neural machine translation,
this paper proposes a comprehensive analysis of adapters for multilingual
speech translation (ST). Starting from different pre-trained models (a
multilingual ST trained on parallel data or a multilingual BART (mBART) trained
on non-parallel multilingual data), we show that adapters can be used to: (a)
efficiently specialize ST to specific language pairs with a low extra cost in
terms of parameters, and (b) transfer from an automatic speech recognition
(ASR) task and an mBART pre-trained model to a multilingual ST task.
Experiments show that adapter tuning offer competitive results to full
fine-tuning, while being much more parameter-efficient.

本文为多语言语音翻译（ST）提供了适配器的完整分析，其中适配器调整在 NLP 中作为微调的高效替代品，可使 ST 针对特定语言对进行专门优化，并从自动语音识别（ASR）任务和预先训练的 mBART 模型中进行转移，同时节省大量参数，具有与完全微调相当的可比结果。

轻量级适配器调整用于多语言语音翻译

Lightweight Adapter Tuning for Multilingual Speech Translation

We introduce dual-decoder Transformer, a new model architecture that jointly
performs automatic speech recognition (ASR) and multilingual speech translation
(ST). Our models are based on the original Transformer architecture (Vaswani et
al., 2017) but consist of two decoders, each responsible for one task (ASR or
ST). Our major contribution lies in how these decoders interact with each
other: one decoder can attend to different information sources from the other
via a dual-attention mechanism. We propose two variants of these architectures
corresponding to two different levels of dependencies between the decoders,
called the parallel and cross dual-decoder Transformers, respectively.
Extensive experiments on the MuST-C dataset show that our models outperform the
previously-reported highest translation performance in the multilingual
settings, and outperform as well bilingual one-to-one results. Furthermore, our
parallel models demonstrate no trade-off between ASR and ST compared to the
vanilla multi-task architecture. Our code and pre-trained models are available
at this https URL

本文介绍了双解码器 Transformer 的新模型架构，该模型同时执行自动语音识别和多语言语音翻译。我们的模型基于原始 Transformer 体系结构，但由两个解码器组成，每个解码器负责一个任务（ASR 或 ST），并通过双关注机制相互交互。我们提出了两种不同的体系结构变体，分别对应于解码器之间的两个不同层次的依赖关系，称为并行双解码器 Transformer 和交叉双解码器 Transformer。我们在 MuST-C 数据集上进行了大量实验，结果显示我们的模型在多语言设置中优于以前报道的最高翻译性能，并且也优于双语一对一结果。此外，我们的并行模型相对于香草多任务体系结构在 ASR 和 ST 之间没有权衡。