Many datasets are biased, namely they contain easy-to-learn features that are
highly correlated with the target class only in the dataset but not in the true
underlying distribution of the data. For this reason, learning unbiased models
from biased data has become a very relevant research topic in the last years.
In this work, we tackle the problem of learning representations that are robust
to biases. We first present a margin-based theoretical framework that allows us
to clarify why recent contrastive losses (InfoNCE, SupCon, etc.) can fail when
dealing with biased data. Based on that, we derive a novel formulation of the
supervised contrastive loss (epsilon-SupInfoNCE), providing more accurate
control of the minimal distance between positive and negative samples.
Furthermore, thanks to our theoretical framework, we also propose FairKL, a new
debiasing regularization loss, that works well even with extremely biased data.
We validate the proposed losses on standard vision datasets including CIFAR10,
CIFAR100, and ImageNet, and we assess the debiasing capability of FairKL with
epsilon-SupInfoNCE, reaching state-of-the-art performance on a number of biased
datasets, including real instances of biases in the wild.

本文提出了一种基于边界的理论框架，并推导出了一种新的有监督对比损失（epsilon-SupInfoNCE）和一种有效的去偏差正则化损失（FairKL），这些方法可以用于从偏差数据中学习无偏模型，并在多个偏差数据集上实现了最先进的性能。