Learning-based representations have become the defacto means to address computer vision tasks. Despite their massive adoption, the amount of work aiming at understanding the internal representations learned by these models is rather limited. Existing methods aimed at model interpretation either require exhaustive manual inspection of visualizations, or link internal network activations with external "possibly useful" annotated concepts. We propose an intermediate scheme in which, given a pretrained model, we automatically identify internal features relevant for the set of classes considered by the model, without requiring additional annotations. We interpret the model through average visualizations of these features. Then, at test time, we explain the network prediction by accompanying the predicted class label with supporting heatmap visualizations derived from the identified relevant features. In addition, we propose a method to address the artifacts introduced by strided operations in deconvnet-based visualizations. Our evaluation on the MNIST, ILSVRC 12 and Fashion 144k datasets quantitatively shows that the proposed method is able to identify relevant internal features for the classes of interest while improving the quality of the produced visualizations.

本文提出了一种基于内部特征及可视化的方法，能够自动识别预先训练模型中与给定类相关的特征，以进行深度模型的解释和阐释，并且提出了一种针对deconvNet可视化操作引入的伪像问题的方法，同时还提出了一个专门用于视觉解释方法客观定量评估的数据集an8Flower。实验证明，该方法在MNIST、ILSVRC12、Fashion144k和an8Flower数据集上能够产生具有良好主题相关特征覆盖的详细解释。

通过解释进行视觉说明: 提高深度神经网络视觉反馈能力