Understanding the trustworthiness of a prediction yielded by a classifier is
critical for the safe and effective use of AI models. Prior efforts have been
proven to be reliable on small-scale datasets. In this work, we study the
problem of predicting trustworthiness on real-world large-scale datasets, where
the task is more challenging due to high-dimensional features, diverse visual
concepts, and large-scale samples. In such a setting, we observe that the
trustworthiness predictors trained with prior-art loss functions, i.e., the
cross entropy loss, focal loss, and true class probability confidence loss, are
prone to view both correct predictions and incorrect predictions to be
trustworthy. The reasons are two-fold. Firstly, correct predictions are
generally dominant over incorrect predictions. Secondly, due to the data
complexity, it is challenging to differentiate the incorrect predictions from
the correct ones on real-world large-scale datasets. To improve the
generalizability of trustworthiness predictors, we propose a novel steep slope
loss to separate the features w.r.t. correct predictions from the ones w.r.t.
incorrect predictions by two slide-like curves that oppose each other. The
proposed loss is evaluated with two representative deep learning models, i.e.,
Vision Transformer and ResNet, as trustworthiness predictors. We conduct
comprehensive experiments and analyses on ImageNet, which show that the
proposed loss effectively improves the generalizability of trustworthiness
predictors. The code and pre-trained trustworthiness predictors for
reproducibility are available at
this https URL

本文研究了在真实世界的大规模数据集中预测可信度的问题，引入了一种新的陡峭斜率损失函数，来提高预测器的泛化能力，并在 ImageNet 数据集上对其进行了实验和分析。