Multilingual machine translation (MMT) benefits from cross-lingual transfer
but is a challenging multitask optimization problem. This is partly because
there is no clear framework to systematically learn language-specific
parameters. Self-supervised learning (SSL) approaches that leverage large
quantities of monolingual data (where parallel data is unavailable) have shown
promise by improving translation performance as complementary tasks to the MMT
task. However, jointly optimizing SSL and MMT tasks is even more challenging.
In this work, we first investigate how to utilize intra-distillation to learn
more *language-specific* parameters and then show the importance of these
language-specific parameters. Next, we propose a novel but simple SSL task,
concurrent denoising, that co-trains with the MMT task by concurrently
denoising monolingual data on both the encoder and decoder. Finally, we apply
intra-distillation to this co-training approach. Combining these two approaches
significantly improves MMT performance, outperforming three state-of-the-art
SSL methods by a large margin, e.g., 11.3\% and 3.7\% improvement on an
8-language and a 15-language benchmark compared with MASS, respectively

本文探讨了如何利用自监督学习的方法，在多语言机器翻译任务中实现参数的有利学习，其中采用了基于噪声消除的简单自监督任务，并将其与原有任务同时进行。通过这两种方法的结合，在 8 种和 15 种语言基准测试中分别比 MASS 等三种最先进的自监督学习方法提高了 11.3％和 3.7％的性能。