Explaining deep convolutional neural networks has been recently drawing
increasing attention since it helps to understand the networks' internal
operations and why they make certain decisions. Saliency maps, which emphasize
salient regions largely connected to the network's decision-making, are one of
the most common ways for visualizing and analyzing deep networks in the
computer vision community. However, saliency maps generated by existing methods
cannot represent authentic information in images due to the unproven proposals
about the weights of activation maps which lack solid theoretical foundation
and fail to consider the relations between each pixel. In this paper, we
develop a novel post-hoc visual explanation method called Shap-CAM based on
class activation mapping. Unlike previous gradient-based approaches, Shap-CAM
gets rid of the dependence on gradients by obtaining the importance of each
pixel through Shapley value. We demonstrate that Shap-CAM achieves better
visual performance and fairness for interpreting the decision making process.
Our approach outperforms previous methods on both recognition and localization
tasks.

本文提出 Shap-CAM，一种基于类激活映射的新型后续可视化解释方法，通过获取 Shapley 值来消除依赖于梯度的先前方法的不足，表现出更好的视觉性能和公平性以解释决策过程。

基于 Shapley 值的卷积神经网络视觉解释

Shap-CAM: Visual Explanations for Convolutional Neural Networks based on  Shapley Value

Recently, increasing attention has been drawn to the internal mechanisms of
convolutional neural networks, and the reason why the network makes specific
decisions. In this paper, we develop a novel post-hoc visual explanation method
called Score-CAM based on class activation mapping. Unlike previous class
activation mapping based approaches, Score-CAM gets rid of the dependence on
gradients by obtaining the weight of each activation map through its forward
passing score on target class, the final result is obtained by a linear
combination of weights and activation maps. We demonstrate that Score-CAM
achieves better visual performance and fairness for interpreting the decision
making process. Our approach outperforms previous methods on both recognition
and localization tasks, it also passes the sanity check. We also indicate its
application as debugging tools. Official code has been released.

本文提出一个名为 Score-CAM 的新型可解释性模型，通过与先前基于类激活映射的方法不同的方式，通过目标类的前向传递得分获得每个激活映射的权重，在权重和激活映射的线性组合下获得最终结果。它获得了更好的视觉性能，良好的公平性，适用于识别和定位任务，以及作为调试工具。该方法摆脱了对梯度的依赖，并通过测试验证。

Score-CAM：卷积神经网络中的得分加权视觉解释

Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural  Networks

To verify and validate networks, it is essential to gain insight into their
decisions, limitations as well as possible shortcomings of training data. In
this work, we propose a post-hoc, optimization based visual explanation method,
which highlights the evidence in the input image for a specific prediction. Our
approach is based on a novel technique to defend against adversarial evidence
(i.e. faulty evidence due to artefacts) by filtering gradients during
optimization. The defense does not depend on human-tuned parameters. It enables
explanations which are both fine-grained and preserve the characteristics of
images, such as edges and colors. The explanations are interpretable, suited
for visualizing detailed evidence and can be tested as they are valid model
inputs. We qualitatively and quantitatively evaluate our approach on a
multitude of models and datasets.

本文提出了一种用于验证神经网络的可视化解释方法，该方法基于优化技术，通过过滤梯度来防止对抗证据。这种方法可以提供细粒度、可解释的图像细节证据，并可针对多个模型和数据集进行定性和定量评估。