Despite substantial progress in applying neural networks (NN) to a wide
variety of areas, they still largely suffer from a lack of transparency and
interpretability. While recent developments in explainable artificial
intelligence attempt to bridge this gap (e.g., by visualizing the correlation
between input pixels and final outputs), these approaches are limited to
explaining low-level relationships, and crucially, do not provide insights on
error correction. In this work, we propose a framework (VRX) to interpret
classification NNs with intuitive structural visual concepts. Given a trained
classification model, the proposed VRX extracts relevant class-specific visual
concepts and organizes them using structural concept graphs (SCG) based on
pairwise concept relationships. By means of knowledge distillation, we show VRX
can take a step towards mimicking the reasoning process of NNs and provide
logical, concept-level explanations for final model decisions. With extensive
experiments, we empirically show VRX can meaningfully answer "why" and "why
not" questions about the prediction, providing easy-to-understand insights
about the reasoning process. We also show that these insights can potentially
provide guidance on improving NN's performance.

本文提出了一个基于结构化视觉概念的框架（VRX），以解释分类神经网络的推理过程，并提供模型决策的逻辑和概念层次的解释，旨在弥补黑盒神经网络缺乏可解释性和透明性的缺陷。通过广泛的实验，研究表明 VRX 可以有效地回答关于预测的 “为什么” 和 “为什么不” 的问题，提供易于理解的洞察，同时还可以提供改善神经网络性能的指导意见。