A rich line of recent work has studied distributionally robust learning
approaches that seek to learn a hypothesis that performs well, in the
worst-case, on many different distributions over a population. We argue that
although the most common approaches seek to minimize the worst-case loss over
distributions, a more reasonable goal is to minimize the worst-case distance to
the true conditional expectation of labels given each covariate. Focusing on
the minmax loss objective can dramatically fail to output a solution minimizing
the distance to the true conditional expectation when certain distributions
contain high levels of label noise. We introduce a new min-max objective based
on what is known as the adversarial moment violation and show that minimizing
this objective is equivalent to minimizing the worst-case $\ell_2$-distance to
the true conditional expectation if we take the adversary's strategy space to
be sufficiently rich. Previous work has suggested minimizing the maximum regret
over the worst-case distribution as a way to circumvent issues arising from
differential noise levels. We show that in the case of square loss, minimizing
the worst-case regret is also equivalent to minimizing the worst-case
$\ell_2$-distance to the true conditional expectation. Although their objective
and our objective both minimize the worst-case distance to the true conditional
expectation, we show that our approach provides large empirical savings in
computational cost in terms of the number of groups, while providing the same
noise-oblivious worst-distribution guarantee as the minimax regret approach,
thus making positive progress on an open question posed by Agarwal and Zhang
(2022).

在分布鲁棒学习中，我们引入了基于对抗性矩违规的新的极小极大目标，并展示了通过最小化该目标等效于最小化与真实条件期望的最坏情况下的 $l_2$ 距离，从而在计算成本上提供了大体量的经验性节省。

分布鲁棒性的瞬间

Taking a Moment for Distributional Robustness

Training with an emphasis on "hard-to-learn" components of the data has been
proven as an effective method to improve the generalization of machine learning
models, especially in the settings where robustness (e.g., generalization
across distributions) is valued. Existing literature discussing this
"hard-to-learn" concept are mainly expanded either along the dimension of the
samples or the dimension of the features. In this paper, we aim to introduce a
simple view merging these two dimensions, leading to a new, simple yet
effective, heuristic to train machine learning models by emphasizing the
worst-cases on both the sample and the feature dimensions. We name our method
W2D following the concept of "Worst-case along Two Dimensions". We validate the
idea and demonstrate its empirical strength over standard benchmarks.

本研究介绍了一种名为 W2D 的机器学习训练方法，该方法通过强调样本和特征维度的最坏情况，提高了模型的泛化能力。实验证明该方法在标准基准测试中具有实际的优势。