Empirical risk minimization (ERM) of neural networks is prone to
over-reliance on spurious correlations and poor generalization on minority
groups. The recent deep feature reweighting (DFR) technique achieves
state-of-the-art group robustness via simple last-layer retraining, but it
requires held-out group and class annotations to construct a group-balanced
reweighting dataset. In this work, we examine this impractical requirement and
find that last-layer retraining can be surprisingly effective with no group
annotations (other than for model selection) and only a handful of class
annotations. We first show that last-layer retraining can greatly improve
worst-group accuracy even when the reweighting dataset has only a small
proportion of worst-group data. This implies a "free lunch" where holding out a
subset of training data to retrain the last layer can substantially outperform
ERM on the entire dataset with no additional data or annotations. To further
improve group robustness, we introduce a lightweight method called selective
last-layer finetuning (SELF), which constructs the reweighting dataset using
misclassifications or disagreements. Our empirical and theoretical results
present the first evidence that model disagreement upsamples worst-group data,
enabling SELF to nearly match DFR on four well-established benchmarks across
vision and language tasks with no group annotations and less than 3% of the
held-out class annotations. Our code is available at
this https URL

通过仅使用模型选择的族群注释和少数类别注释，最后一层的重新训练可以显著提高最差群组的准确性，而无需额外的数据或注释。此外，我们还引入了一种轻量级方法，即选择性的最后一层微调（SELF），通过使用误分类或不一致性的数据构建了重新加权数据集，实现了在视觉和语言任务的四个基准测试中无需族群注释和少于 3％的保留类别注释即可与 DFR 相媲美。

面向少量注释的群体鲁棒性的最后一层重新训练

Towards Last-layer Retraining for Group Robustness with Fewer  Annotations

Models trained with empirical risk minimization (ERM) are known to learn to
rely on spurious features, i.e., their prediction is based on undesired
auxiliary features which are strongly correlated with class labels but lack
causal reasoning. This behavior particularly degrades accuracy in groups of
samples of the correlated class that are missing the spurious feature or
samples of the opposite class but with the spurious feature present. The
recently proposed Deep Feature Reweighting (DFR) method improves accuracy of
these worst groups. Based on the main argument that ERM mods can learn core
features sufficiently well, DFR only needs to retrain the last layer of the
classification model with a small group-balanced data set. In this work, we
examine the applicability of DFR to realistic data in the medical domain.
Furthermore, we investigate the reasoning behind the effectiveness of
last-layer retraining and show that even though DFR has the potential to
improve the accuracy of the worst group, it remains susceptible to spurious
correlations.

通过对医学领域的真实数据进行研究，我们考察了深度特征重新加权方法（DFR）对于应用的可行性，并研究了最后一层重训练方法背后的有效性原因，结果表明 DFR 可以提高最差群体的准确性，但仍对虚假相关性敏感。