Multilingual machine translation has attracted much attention recently due to
its support of knowledge transfer among languages and the low cost of training
and deployment compared with numerous bilingual models. A known challenge of
multilingual models is the negative language interference. In order to enhance
the translation quality, deeper and wider architectures are applied to
multilingual modeling for larger model capacity, which suffers from the
increased inference cost at the same time. It has been pointed out in recent
studies that parameters shared among languages are the cause of interference
while they may also enable positive transfer. Based on these insights, we
propose an adaptive and sparse architecture for multilingual modeling, and
train the model to learn shared and language-specific parameters to improve the
positive transfer and mitigate the interference. The sparse architecture only
activates a sub-network which preserves inference efficiency, and the adaptive
design selects different sub-networks based on the input languages. Our model
outperforms strong baselines across multiple benchmarks. On the large-scale
OPUS dataset with $100$ languages, we achieve $+2.1$, $+1.3$ and $+6.2$ BLEU
improvements in one-to-many, many-to-one and zero-shot tasks respectively
compared to standard Transformer without increasing the inference cost.

通过采用自适应的、稀疏的多语言建模结构，实现共享参数和语言特定参数的学习，提高正向转移和减轻干扰，从而达到改善翻译质量和保持推理效率的目的。与标准 Transformer 相比，我们在多项基准测试中的表现都超过了强基准，特别是在使用 100 种语言的大规模 OPUS 数据集时，对于一对多、多对一和零 shot 任务分别提高了 2.1、1.3 和 6.2 BLEU 分数，未增加推理成本。