There has been a significant surge of interest recently around the concept of explainable artificial intelligence (XAI), where the goal is to produce an interpretation for a decision made by a machine learning algorithm. Of particular interest is the interpretation of how deep neural networks make decisions, given the complexity and `black box' nature of such networks. Given the infancy of the field, there has been very limited exploration into the assessment of the performance of explainability methods, with most evaluations centered around subjective visual interpretation of the produced interpretations. In this study, we explore a more machine-centric strategy for quantifying the performance of explainability methods on deep neural networks via the notion of decision-making impact analysis. We introduce two quantitative performance metrics: i) Impact Score, which assesses the percentage of critical factors with either strong confidence reduction impact or decision changing impact, and ii) Impact Coverage, which assesses the percentage coverage of adversarially impacted factors in the input. A comprehensive analysis using this approach was conducted on several state-of-the-art explainability methods (LIME, SHAP, Expected Gradients, GSInquire) on a ResNet-50 deep convolutional neural network using a subset of ImageNet for the task of image classification. Experimental results show that the critical regions identified by LIME within the tested images had the lowest impact on the decision-making process of the network (~38%), with progressive increase in decision-making impact for SHAP (~44%), Expected Gradients (~51%), and GSInquire (~76%). While by no means perfect, the hope is that the proposed machine-centric strategy helps push the conversation forward towards better metrics for evaluating explainability methods and improve trust in deep neural networks.

本次研究使用决策影响分析的方法，提出了两个指标来量化深度神经网络的可解释性方法的性能，并对几种最先进的解释性方法（LIME，SHAP，Expected Gradients，GSInquire）在 ResNet-50 深度卷积神经网络上进行了全面分析。实验结果表明，受测试图像中由 LIME 确定的关键区域对网络的决策过程的影响最小，SHAP，Expected Gradients 和 GSInquire 的影响逐渐增加，提出的机器-中心策略有助于推进评估可解释性方法的更好度量和提高深度神经网络的信任度。

解释反映决策吗？一种以机器为中心的策略来量化可解释性算法的性能