In this work, we study how the generalization performance of a given
direction changes with its sampling ratio in Multilingual Neural Machine
Translation (MNMT). By training over 200 multilingual models with various model
sizes, directions, and total numbers of tasks, we find that scalarization leads
to a multitask trade-off front that deviates from the traditional Pareto front
when there exists data imbalance in the training corpus. That is, the
performance of certain translation directions does not improve with the
increase of its weight in the multi-task optimization objective, which poses
greater challenge to improve the overall performance of all directions. Based
on our observations, we propose the Double Power Law to predict the unique
performance trade-off front in MNMT, which is robust across various languages,
data adequacy and number of tasks. Finally, we formulate sample ratio selection
in MNMT as an optimization problem based on the Double Power Law, which
achieves better performance than temperature searching and gradient
manipulation methods using up to half of the total training budget in our
experiments.

本研究旨在探究在多语言神经机器翻译中，给定方向的泛化性能如何随着其采样比例的变化而改变。我们发现，当训练语料具有数据不平衡时，标量化会导致多任务折衷前沿偏离传统的 Pareto 前沿。最后，我们基于观察结果提出了双重幂律，用于预测 MNMT 中的独特性能折衷前沿，并将 MNMT 中的采样比例选择问题制定为基于双重幂律的优化问题，该方法在实验中表现优异。