Gaining insight into how deep convolutional neural network models perform
image classification and how to explain their outputs have been a concern to
computer vision researchers and decision makers. These deep models are often
referred to as black box due to low comprehension of their internal workings.
As an effort to developing explainable deep learning models, several methods
have been proposed such as finding gradients of class output with respect to
input image (sensitivity maps), class activation map (CAM), and Gradient based
Class Activation Maps (Grad-CAM). These methods under perform when localizing
multiple occurrences of the same class and do not work for all CNNs. In
addition, Grad-CAM does not capture the entire object in completeness when used
on single object images, this affect performance on recognition tasks. With the
intention to create an enhanced visual explanation in terms of visual
sharpness, object localization and explaining multiple occurrences of objects
in a single image, we present Smooth Grad-CAM++ \footnote{Simple demo:
this http URL}, a technique that combines methods from two other
recent techniques---SMOOTHGRAD and Grad-CAM++. Our Smooth Grad-CAM++ technique
provides the capability of either visualizing a layer, subset of feature maps,
or subset of neurons within a feature map at each instance at the inference
level (model prediction process). After experimenting with few images, Smooth
Grad-CAM++ produced more visually sharp maps with better localization of
objects in the given input images when compared with other methods.

本文介绍了一种叫做 Smooth Grad-CAM++ 的新方法，用于可视化神经网络内部的决策过程并解释其输出，该方法在多种场合下表现出更优秀的效果，特别是在物体定位和图像分类方面。

Smooth Grad-CAM++: 深度卷积神经网络模型增强推理可视化技术

Smooth Grad-CAM++: An Enhanced Inference Level Visualization Technique  for Deep Convolutional Neural Network Models

Over the last decade, Convolutional Neural Network (CNN) models have been
highly successful in solving complex vision problems. However, these deep
models are perceived as "black box" methods considering the lack of
understanding of their internal functioning. There has been a significant
recent interest in developing explainable deep learning models, and this paper
is an effort in this direction. Building on a recently proposed method called
Grad-CAM, we propose a generalized method called Grad-CAM++ that can provide
better visual explanations of CNN model predictions, in terms of better object
localization as well as explaining occurrences of multiple object instances in
a single image, when compared to state-of-the-art. We provide a mathematical
derivation for the proposed method, which uses a weighted combination of the
positive partial derivatives of the last convolutional layer feature maps with
respect to a specific class score as weights to generate a visual explanation
for the corresponding class label. Our extensive experiments and evaluations,
both subjective and objective, on standard datasets showed that Grad-CAM++
provides promising human-interpretable visual explanations for a given CNN
architecture across multiple tasks including classification, image caption
generation and 3D action recognition; as well as in new settings such as
knowledge distillation.

该论文介绍了一个名为 Grad-CAM++ 的泛化方法，它以更好的对象定位和解释单个图像中多个对象实例的方式提供了更好的 CNN 模型预测视觉解释，并在标准数据集上进行了广泛的实验和评估，包括分类，图像字幕生成和 3D 动作识别等多个任务。