Multilingual machine translation (MMT), trained on a mixture of parallel and
monolingual data, is key for improving translation in low-resource language
pairs. However, the literature offers conflicting results on the performance of
different methods. To resolve this, we examine how denoising autoencoding (DAE)
and backtranslation (BT) impact MMT under different data conditions and model
scales. Unlike prior studies, we use a realistic dataset of 100 directions and
consider many domain combinations of monolingual and test data. We find that
monolingual data generally helps MMT, but models are surprisingly brittle to
domain mismatches, especially at smaller model scales. BT is beneficial when
the parallel, monolingual, and test data sources are similar but can be
detrimental otherwise, while DAE is less effective than previously reported.
Next, we analyze the impact of scale (from 90M to 1.6B parameters) and find it
is important for both methods, particularly DAE. As scale increases, DAE
transitions from underperforming the parallel-only baseline at 90M to
converging with BT performance at 1.6B, and even surpassing it in low-resource.
These results offer new insights into how to best use monolingual data in MMT.

通过实验研究了多种语言机器翻译在不同数据条件和模型尺度下，去噪自编码器和回译技术对多种语言机器翻译的影响、以及单语数据在多种语言机器翻译中的作用和数据规模对机器翻译的影响等问题。结果发现单语数据总体有益于机器翻译，但在小型模型中对领域不匹配，BT 对相似的平行，单语和测试数据来源是有益的，但在其他情况下会有不利影响，DAE 相对于以前报告的效果不如。规模永远是关键。

单语数据何时对多语言翻译有帮助：领域和模型规模的作用

When Does Monolingual Data Help Multilingual Translation: The Role of  Domain and Model Scale

The dependency of the generalization error of neural networks on model and
dataset size is of critical importance both in practice and for understanding
the theory of neural networks. Nevertheless, the functional form of this
dependency remains elusive. In this work, we present a functional form which
approximates well the generalization error in practice. Capitalizing on the
successful concept of model scaling (e.g., width, depth), we are able to
simultaneously construct such a form and specify the exact models which can
attain it across model/data scales. Our construction follows insights obtained
from observations conducted over a range of model/data scales, in various model
types and datasets, in vision and language tasks. We show that the form both
fits the observations well across scales, and provides accurate predictions
from small- to large-scale models and data.

本论文提出基于模型缩放的方法来构建适合各类模型和数据规模的函数形式，针对神经网络的泛化误差进行观测并给出了精确预测。