Neural networks trained with ERM (empirical risk minimization) sometimes
learn unintended decision rules, in particular when their training data is
biased, i.e., when training labels are strongly correlated with undesirable
features. To prevent a network from learning such features, recent methods
augment training data such that examples displaying spurious correlations
(i.e., bias-aligned examples) become a minority, whereas the other,
bias-conflicting examples become prevalent. However, these approaches are
sometimes difficult to train and scale to real-world data because they rely on
generative models or disentangled representations. We propose an alternative
based on mixup, a popular augmentation that creates convex combinations of
training examples. Our method, coined SelecMix, applies mixup to contradicting
pairs of examples, defined as showing either (i) the same label but dissimilar
biased features, or (ii) different labels but similar biased features.
Identifying such pairs requires comparing examples with respect to unknown
biased features. For this, we utilize an auxiliary contrastive model with the
popular heuristic that biased features are learned preferentially during
training. Experiments on standard benchmarks demonstrate the effectiveness of
the method, in particular when label noise complicates the identification of
bias-conflicting examples.

提出一种新的数据增广方法 SelecMix，旨在解决神经网络训练中数据偏差带来的问题，并在标准基准测试中证明了其有效性，尤其是在标签噪声情况下。