Understanding intermediate layers of a deep learning model and discovering
the driving features of stimuli have attracted much interest, recently.
Explainable artificial intelligence (XAI) provides a new way to open an AI
black box and makes a transparent and interpretable decision. This paper
proposes a new explainable convolutional neural network (XCNN) which represents
important and driving visual features of stimuli in an end-to-end model
architecture. This network employs encoder-decoder neural networks in a CNN
architecture to represent regions of interest in an image based on its
category. The proposed model is trained without localization labels and
generates a heat-map as part of the network architecture without extra
post-processing steps. The experimental results on the CIFAR-10, Tiny ImageNet,
and MNIST datasets showed the success of our algorithm (XCNN) to make CNNs
explainable. Based on visual assessment, the proposed model outperforms the
current algorithms in class-specific feature representation and interpretable
heatmap generation while providing a simple and flexible network architecture.
The initial success of this approach warrants further study to enhance weakly
supervised localization and semantic segmentation in explainable frameworks.

本文介绍了一种新的可解释卷积神经网络（XCNN），能够表示视觉刺激的重要和驱动特征，并提供了无需本地化标签和额外后处理步骤的热图生成。在多个数据集上的实验证明了 XCNN 的成功，验证了其在类别特定特征表示和可解释热图生成方面的性能优于当前算法，同时提供了简单灵活的网络结构，为提高可解释的弱监督定位和语义分割奠定了基础。