Scalarization is a general technique that can be deployed in any
multiobjective setting to reduce multiple objectives into one, such as recently
in RLHF for training reward models that align human preferences. Yet some have
dismissed this classical approach because linear scalarizations are known to
miss concave regions of the Pareto frontier. To that end, we aim to find simple
non-linear scalarizations that can explore a diverse set of $k$ objectives on
the Pareto frontier, as measured by the dominated hypervolume. We show that
hypervolume scalarizations with uniformly random weights are surprisingly
optimal for provably minimizing the hypervolume regret, achieving an optimal
sublinear regret bound of $O(T^{-1/k})$, with matching lower bounds that
preclude any algorithm from doing better asymptotically. As a theoretical case
study, we consider the multiobjective stochastic linear bandits problem and
demonstrate that by exploiting the sublinear regret bounds of the hypervolume
scalarizations, we can derive a novel non-Euclidean analysis that produces
improved hypervolume regret bounds of $\tilde{O}( d T^{-1/2} + T^{-1/k})$. We
support our theory with strong empirical performance of using simple
hypervolume scalarizations that consistently outperforms both the linear and
Chebyshev scalarizations, as well as standard multiobjective algorithms in
bayesian optimization, such as EHVI.

使用综合性标量化技术来探索 Pareto 前沿上的多样化目标集合，证明均匀随机权重的超体积标量化方法在减小超体积遗憾方面是最优的，并在多目标随机线性赌博机问题中得出新的非欧几里得分析结果，获得了改进的超体积遗憾界限。

次线性超体积遗憾的最佳标量化方法

Optimal Scalarizations for Sublinear Hypervolume Regret

In this work, we study how the generalization performance of a given
direction changes with its sampling ratio in Multilingual Neural Machine
Translation (MNMT). By training over 200 multilingual models with various model
sizes, directions, and total numbers of tasks, we find that scalarization leads
to a multitask trade-off front that deviates from the traditional Pareto front
when there exists data imbalance in the training corpus. That is, the
performance of certain translation directions does not improve with the
increase of its weight in the multi-task optimization objective, which poses
greater challenge to improve the overall performance of all directions. Based
on our observations, we propose the Double Power Law to predict the unique
performance trade-off front in MNMT, which is robust across various languages,
data adequacy and number of tasks. Finally, we formulate sample ratio selection
in MNMT as an optimization problem based on the Double Power Law, which
achieves better performance than temperature searching and gradient
manipulation methods using up to half of the total training budget in our
experiments.

本研究旨在探究在多语言神经机器翻译中，给定方向的泛化性能如何随着其采样比例的变化而改变。我们发现，当训练语料具有数据不平衡时，标量化会导致多任务折衷前沿偏离传统的 Pareto 前沿。最后，我们基于观察结果提出了双重幂律，用于预测 MNMT 中的独特性能折衷前沿，并将 MNMT 中的采样比例选择问题制定为基于双重幂律的优化问题，该方法在实验中表现优异。

多语言神经机器翻译的 Pareto 前沿

On the Pareto Front of Multilingual Neural Machine Translation

Recent multi-task learning research argues against unitary scalarization,
where training simply minimizes the sum of the task losses. Several ad-hoc
multi-task optimization algorithms have instead been proposed, inspired by
various hypotheses about what makes multi-task settings difficult. The majority
of these optimizers require per-task gradients, and introduce significant
memory, runtime, and implementation overhead. We show that unitary
scalarization, coupled with standard regularization and stabilization
techniques from single-task learning, matches or improves upon the performance
of complex multi-task optimizers in popular supervised and reinforcement
learning settings. We then present an analysis suggesting that many specialized
multi-task optimizers can be partly interpreted as forms of regularization,
potentially explaining our surprising results. We believe our results call for
a critical reevaluation of recent research in the area.

通过标量化，加标准正则化和稳定技巧，我们可以在流行的监督和强化学习环境中，匹配甚至优于复杂多任务优化器的性能，我们认为这种结果需要对最近的多任务学习研究进行关键重新评估。