We develop a technique for automatically detecting the classification errors of a pre-trained visual classifier. Our method is agnostic to the form of the classifier, requiring access only to classifier responses to a set of inputs. We train a parametric binary classifier (error/correct) on a representation derived from a set of classifier responses generated from multiple copies of the same input, each subject to a different natural image transformation. Thus, we establish a measure of confidence in classifier's decision by analyzing the invariance of its decision under various transformations. In experiments with multiple data sets (STL-10,CIFAR-100,ImageNet) and classifiers, we demonstrate new state of the art for the error detection task. In addition, we apply our technique to novelty detection scenarios, where we also demonstrate state of the art results.

本文介绍了一种自动检测预先训练的视觉分类器中分类错误的技术，该技术是对分类器形式不可知的，并且仅需要访问对一组输入的分类器响应。我们使用由来源于同一输入的多个分类器响应生成的表示的参数二元分类器(错误/正确)进行训练，每个分类器都受不同的自然图像变换的影响。因此，我们通过分析分类器在各种转换下的不变性来建立分类器决策的置信度的度量。在多个数据集（STL-10、CIFAR-100、ImageNet）和分类器的实验中，我们展示了错误检测任务的新的最先进结果。此外，我们将该技术应用于新颖性检测场景，并展示了最先进的结果。

通过图像变形的不变性获得信心