Traditional multilingual neural machine translation (MNMT) uses a single
model to translate all directions. However, with the increasing scale of
language pairs, simply using a single model for massive MNMT brings new
challenges: parameter tension and large computations. In this paper, we revisit
multi-way structures by assigning an individual branch for each language
(group). Despite being a simple architecture, it is challenging to train
de-centralized models due to the lack of constraints to align representations
from all languages. We propose a localized training recipe to map different
branches into a unified space, resulting in an efficient detachable model,
Lego-MT. For a fair comparison, we collect data from OPUS and build the first
large-scale open-source translation benchmark covering 7 language-centric data,
each containing 445 language pairs. Experiments show that Lego-MT (1.2B) brings
gains of more than 4 BLEU while outperforming M2M-100 (12B) (We will public all
training data, models, and checkpoints)

本文提出了一种名为 Lego-MT 的多向分支神经机器翻译模型，通过局部化训练，将不同的分支映射到一个统一的空间中，达到高效、可分离的效果，并在收集的包含 7 个语言中心数据的大型开源翻译基准测试中，取得了超过 4 个 BLEU 的提升，优于 M2M-100 模型。

Lego-MT: 面向大规模多语机器翻译的可拆卸模型

Lego-MT: Towards Detachable Models in Massively Multilingual Machine Translation

Despite the great success of NMT, there still remains a severe challenge: it
is hard to interpret the internal dynamics during its training process. In this
paper we propose to understand learning dynamics of NMT by using a recent
proposed technique named Loss Change Allocation
(LCA)~\citep{lan-2019-loss-change-allocation}. As LCA requires calculating the
gradient on an entire dataset for each update, we instead present an
approximate to put it into practice in NMT scenario. %motivated by the lesson
from sgd. Our simulated experiment shows that such approximate calculation is
efficient and is empirically proved to deliver consistent results to the
brute-force implementation. In particular, extensive experiments on two
standard translation benchmark datasets reveal some valuable findings.

该研究旨在通过使用最新提出的技术 LCA 了解 NMT 的学习动态，提出了一种在 NMT 场景中实现 LCA 的近似方法，模拟实验证明此方法是有效的并能够在两个标准的翻译基准数据集上提供有价值的发现。