Subpopulation shift exists widely in many real-world applications, which
refers to the training and test distributions that contain the same
subpopulation groups but with different subpopulation proportions. Ignoring
subpopulation shifts may lead to significant performance degradation and
fairness concerns. Importance reweighting is a classical and effective way to
handle the subpopulation shift. However, recent studies have recognized that
most of these approaches fail to improve the performance especially when
applied to over-parameterized neural networks which are capable of fitting any
training samples. In this work, we propose a simple yet practical framework,
called reweighted mixup (RMIX), to mitigate the overfitting issue in
over-parameterized models by conducting importance weighting on the ''mixed''
samples. Benefiting from leveraging reweighting in mixup, RMIX allows the model
to explore the vicinal space of minority samples more, thereby obtaining more
robust model against subpopulation shift. When the subpopulation memberships
are unknown, the training-trajectories-based uncertainty estimation is equipped
in the proposed RMIX to flexibly characterize the subpopulation distribution.
We also provide insightful theoretical analysis to verify that RMIX achieves
better generalization bounds over prior works. Further, we conduct extensive
empirical studies across a wide range of tasks to validate the effectiveness of
the proposed method.

提出一种名为 RMIX 的简单实用的框架来解决超参数化神经网络中意义重要性加权方法效果不佳的问题，从而更好地应对含不同子群比例的训练和测试分布的子群转移问题，并通过混合样本上的重要性加权来获取更鲁棒的模型。同时，RMIX 还利用训练轨迹的不确定性估计来灵活地表征子群分布，并通过理论分析验证了其具有更好的广义化界限。

基于混合重赋权的子种群转移方法

Reweighted Mixup for Subpopulation Shift

Machine learning models often perform poorly on subgroups that are
underrepresented in the training data. Yet, little is understood on the
variation in mechanisms that cause subpopulation shifts, and how algorithms
generalize across such diverse shifts at scale. In this work, we provide a
fine-grained analysis of subpopulation shift. We first propose a unified
framework that dissects and explains common shifts in subgroups. We then
establish a comprehensive benchmark of 20 state-of-the-art algorithms evaluated
on 12 real-world datasets in vision, language, and healthcare domains. With
results obtained from training over 10,000 models, we reveal intriguing
observations for future progress in this space. First, existing algorithms only
improve subgroup robustness over certain types of shifts but not others.
Moreover, while current algorithms rely on group-annotated validation data for
model selection, we find that a simple selection criterion based on worst-class
accuracy is surprisingly effective even without any group information. Finally,
unlike existing works that solely aim to improve worst-group accuracy (WGA), we
demonstrate the fundamental tradeoff between WGA and other important metrics,
highlighting the need to carefully choose testing metrics. Code and data are
available at: this https URL.

通过实验测试，我们对机器学习中的次群体转变机制和算法的泛化能力进行了深入分析，并提出了一种基于最差类准确率的简单选择标准，强调了最差组准确率和其他重要指标之间的基本权衡关系。

变化十分困难：分析亚群体转变

Change is Hard: A Closer Look at Subpopulation Shift

Many machine learning tasks involve subpopulation shift where the testing
data distribution is a subpopulation of the training distribution. For such
settings, a line of recent work has proposed the use of a variant of empirical
risk minimization(ERM) known as distributionally robust optimization (DRO). In
this work, we apply DRO to real, large-scale tasks with subpopulation shift,
and observe that DRO performs relatively poorly, and moreover has severe
instability. We identify one direct cause of this phenomenon: sensitivity of
DRO to outliers in the datasets. To resolve this issue, we propose the
framework of DORO, for Distributional and Outlier Robust Optimization. At the
core of this approach is a refined risk function which prevents DRO from
overfitting to potential outliers. We instantiate DORO for the Cressie-Read
family of R\'enyi divergence, and delve into two specific instances of this
family: CVaR and $\chi^2$-DRO. We theoretically prove the effectiveness of the
proposed method, and empirically show that DORO improves the performance and
stability of DRO with experiments on large modern datasets, thereby positively
addressing the open question raised by Hashimoto et al., 2018.

提出了一种名为 DORO 的分布与离群点鲁棒优化框架，其中核心在于改进的风险函数，可以解决分布变化和离群点的问题，从而提高了机器学习的性能和稳定性。该方法有助于提升现代大型数据集的实验结果。

DORO：分布式和异常值鲁棒优化

DORO: Distributional and Outlier Robust Optimization

We develop a methodology for assessing the robustness of models to
subpopulation shift---specifically, their ability to generalize to novel data
subpopulations that were not observed during training. Our approach leverages
the class structure underlying existing datasets to control the data
subpopulations that comprise the training and test distributions. This enables
us to synthesize realistic distribution shifts whose sources can be precisely
controlled and characterized, within existing large-scale datasets. Applying
this methodology to the ImageNet dataset, we create a suite of subpopulation
shift benchmarks of varying granularity. We then validate that the
corresponding shifts are tractable by obtaining human baselines for them.
Finally, we utilize these benchmarks to measure the sensitivity of standard
model architectures as well as the effectiveness of off-the-shelf train-time
robustness interventions. Code and data available at
this https URL .

本文提出一种方法，通过控制数据次群体的训练和测试集来实现对模型对子群体变化的鲁棒性进行评估和测试，并在 ImageNet 数据集上得到了验证。