Leveraging the models' outputs, specifically the logits, is a common approach
to estimating the test accuracy of a pre-trained neural network on
out-of-distribution (OOD) samples without requiring access to the corresponding
ground truth labels. Despite their ease of implementation and computational
efficiency, current logit-based methods are vulnerable to overconfidence
issues, leading to prediction bias, especially under the natural shift. In this
work, we first study the relationship between logits and generalization
performance from the view of low-density separation assumption. Our findings
motivate our proposed method MaNo which (1) applies a data-dependent
normalization on the logits to reduce prediction bias, and (2) takes the $L_p$
norm of the matrix of normalized logits as the estimation score. Our
theoretical analysis highlights the connection between the provided score and
the model's uncertainty. We conduct an extensive empirical study on common
unsupervised accuracy estimation benchmarks and demonstrate that MaNo achieves
state-of-the-art performance across various architectures in the presence of
synthetic, natural, or subpopulation shifts.

我们提出了一种基于 logits 的方法 MaNo，通过对 logits 进行数据依赖的归一化来减少预测偏差，并将归一化 logits 的矩阵的 Lp 范数作为评估分数，我们在多个架构上进行了广泛的实证研究并证明 MaNo 在合成、自然或亚种群变化下实现了最先进的性能。

MANO：利用矩阵范数在分布偏移下进行无监督准确度估计

MANO: Exploiting Matrix Norm for Unsupervised Accuracy Estimation Under  Distribution Shifts

Detecting out-of-distribution (OOD) samples are crucial for machine learning
models deployed in open-world environments. Classifier-based scores are a
standard approach for OOD detection due to their fine-grained detection
capability. However, these scores often suffer from overconfidence issues,
misclassifying OOD samples distant from the in-distribution region. To address
this challenge, we propose a method called Nearest Neighbor Guidance (NNGuide)
that guides the classifier-based score to respect the boundary geometry of the
data manifold. NNGuide reduces the overconfidence of OOD samples while
preserving the fine-grained capability of the classifier-based score. We
conduct extensive experiments on ImageNet OOD detection benchmarks under
diverse settings, including a scenario where the ID data undergoes natural
distribution shift. Our results demonstrate that NNGuide provides a significant
performance improvement on the base detection scores, achieving
state-of-the-art results on both AUROC, FPR95, and AUPR metrics. The code is
given at https://github.com/roomo7time/nnguide.

检测开放环境中机器学习模型的异常样本是至关重要的。通过引入最近邻引导（NNGuide）方法，我们解决了由于边界几何导致的分类器评分过分自信的问题，同时保持了分类器评分的细粒度能力，并在多种设置下，包括 ID 数据自然分布变化的情况下，在 ImageNet 异常检测基准测试中取得了显著的性能提升，达到了 AUROC、FPR95 和 AUPR 指标的最佳结果。