Label smoothing (LS) is a popular regularisation method for training deep
neural network classifiers due to its effectiveness in improving test accuracy
and its simplicity in implementation. "Hard" one-hot labels are "smoothed" by
uniformly distributing probability mass to other classes, reducing overfitting.
In this work, we reveal that LS negatively affects selective classification
(SC) - where the aim is to reject misclassifications using a model's predictive
uncertainty. We first demonstrate empirically across a range of tasks and
architectures that LS leads to a consistent degradation in SC. We then explain
this by analysing logit-level gradients, showing that LS exacerbates
overconfidence and underconfidence by regularising the max logit more when the
probability of error is low, and less when the probability of error is high.
This elucidates previously reported experimental results where strong
classifiers underperform in SC. We then demonstrate the empirical effectiveness
of logit normalisation for recovering lost SC performance caused by LS.
Furthermore, based on our gradient analysis, we explain why such normalisation
is effective. We will release our code shortly.

标签平滑是一种用于训练深度神经网络分类器的流行的正则化方法，它通过均匀分布概率质量到其他类别来减少过拟合。然而，本研究揭示了标签平滑对选择性分类产生负面影响，并通过分析逻辑斜率梯度解释了这一现象。我们发现，采用逻辑斜率归一化的方法可以恢复由于标签平滑引起的选择性分类性能损失。