Detecting out-of-distribution inputs is critical for safe deployment of
machine learning models in the real world. However, neural networks are known
to suffer from the overconfidence issue, where they produce abnormally high
confidence for both in- and out-of-distribution inputs. In this work, we show
that this issue can be mitigated through Logit Normalization (LogitNorm) -- a
simple fix to the cross-entropy loss -- by enforcing a constant vector norm on
the logits in training. Our method is motivated by the analysis that the norm
of the logit keeps increasing during training, leading to overconfident output.
Our key idea behind LogitNorm is thus to decouple the influence of output's
norm during network optimization. Trained with LogitNorm, neural networks
produce highly distinguishable confidence scores between in- and
out-of-distribution data. Extensive experiments demonstrate the superiority of
LogitNorm, reducing the average FPR95 by up to 42.30% on common benchmarks.

本文提出 Logit Normalization 方法，通过约束神经网络输出的 logits 向量的范数，以减少 out-of-distribution 数据的高置信度问题，并在常见基准测试中取得高度可分辨的置信度分数。