We propose a novel confidence scoring mechanism for deep neural networks
based on a two-model paradigm involving a base model and a meta-model. The
confidence score is learned by the meta-model observing the base model
succeeding/failing at its task. As features to the meta-model, we investigate
linear classifier probes inserted between the various layers of the base model.
Our experiments demonstrate that this approach outperforms various baselines in
a filtering task, i.e., task of rejecting samples with low confidence.
Experimental results are presented using CIFAR-10 and CIFAR-100 dataset with
and without added noise. We discuss the importance of confidence scoring to
bridge the gap between experimental and real-world applications.

本文提出了一种基于两个模型范式的深度神经网络置信度评分机制，通过在任务中观察基础模型的成功 / 失败来学习元模型的置信度评分，实验结果表明该方法在过滤任务中优于各种基线，并使用带或不带噪声的 CIFAR-10 和 CIFAR-100 数据集进行了实验研究，讨论了置信度评分在实验和实际应用之间的重要性。