Selective classification, in which models can abstain on uncertain
predictions, is a natural approach to improving accuracy in settings where
errors are costly but abstentions are manageable. In this paper, we find that
while selective classification can improve average accuracies, it can
simultaneously magnify existing accuracy disparities between various groups
within a population, especially in the presence of spurious correlations. We
observe this behavior consistently across five vision and NLP datasets.
Surprisingly, increasing abstentions can even decrease accuracies on some
groups. To better understand this phenomenon, we study the margin distribution,
which captures the model's confidences over all predictions. For symmetric
margin distributions, we prove that whether selective classification
monotonically improves or worsens accuracy is fully determined by the accuracy
at full coverage (i.e., without any abstentions) and whether the distribution
satisfies a property we call left-log-concavity. Our analysis also shows that
selective classification tends to magnify full-coverage accuracy disparities.
Motivated by our analysis, we train distributionally-robust models that achieve
similar full-coverage accuracies across groups and show that selective
classification uniformly improves each group on these models. Altogether, our
results suggest that selective classification should be used with care and
underscore the importance of training models to perform equally well across
groups at full coverage.

本文研究了分类模型的选择性分类方法，发现在存在虚假相关性时，选择性分类可以提高平均准确度，但会放大不同群体之间的准确度差异。我们的分析表明，对于对称边缘分布，选择性分类是通过完全覆盖（即没有犹豫）下的准确度和分布是否满足左对数凹性质来确定改善或恶化准确度。基于我们的分析，我们训练了分布稳健模型，并表明选择性分类可以在这些模型上统一改进每个群体的准确性。