Adapters have emerged as a modular and parameter-efficient approach to
(zero-shot) cross-lingual transfer. The established MAD-X framework employs
separate language and task adapters which can be arbitrarily combined to
perform the transfer of any task to any target language. Subsequently, BAD-X,
an extension of the MAD-X framework, achieves improved transfer at the cost of
MAD-X's modularity by creating "bilingual" adapters specific to the
source-target language pair. In this work, we aim to take the best of both
worlds by (i) fine-tuning task adapters adapted to the target language(s)
(so-called "target language-ready" (TLR) adapters) to maintain high transfer
performance, but (ii) without sacrificing the highly modular design of MAD-X.
The main idea of "target language-ready" adapters is to resolve the
training-vs-inference discrepancy of MAD-X: the task adapter "sees" the target
language adapter for the very first time during inference, and thus might not
be fully compatible with it. We address this mismatch by exposing the task
adapter to the target language adapter during training, and empirically
validate several variants of the idea: in the simplest form, we alternate
between using the source and target language adapters during task adapter
training, which can be generalized to cycling over any set of language
adapters. We evaluate different TLR-based transfer configurations with varying
degrees of generality across a suite of standard cross-lingual benchmarks, and
find that the most general (and thus most modular) configuration consistently
outperforms MAD-X and BAD-X on most tasks and languages.

本文旨在通过 fine-tuning 目标语言准备（TLR）适配器来取得高传输性能，同时不牺牲 MAD-X 的高度模块化设计。