Image classifiers are typically scored on their test set accuracy, but high
accuracy can mask a subtle type of model failure. We find that high scoring
convolutional neural networks (CNNs) on popular benchmarks exhibit troubling
pathologies that allow them to display high accuracy even in the absence of
semantically salient features. When a model provides a high-confidence decision
without salient supporting input features, we say the classifier has
overinterpreted its input, finding too much class-evidence in patterns that
appear nonsensical to humans. Here, we demonstrate that neural networks trained
on CIFAR-10 and ImageNet suffer from overinterpretation, and we find models on
CIFAR-10 make confident predictions even when 95% of input images are masked
and humans cannot discern salient features in the remaining pixel-subsets. We
introduce Batched Gradient SIS, a new method for discovering sufficient input
subsets for complex datasets, and use this method to show the sufficiency of
border pixels in ImageNet for training and testing. Although these patterns
portend potential model fragility in real-world deployment, they are in fact
valid statistical patterns of the benchmark that alone suffice to attain high
test accuracy. Unlike adversarial examples, overinterpretation relies upon
unmodified image pixels. We find ensembling and input dropout can each help
mitigate overinterpretation.

该研究发现，高得分的卷积神经网络在流行的基准测试中显示出令人担忧的病理现象，使它们即使在缺乏语义明显特征的情况下也能显示高准确度，这被称为过度解释其输入。研究提出了一种新的方法来发现复杂数据集的足够输入子集，并使用该方法显示 ImageNet 中边缘像素的足够性进行训练和测试，结果表明集成学习和输入 dropout 可以缓解过度解释现象。