Concept-based explanations permit to understand the predictions of a deep
neural network (DNN) through the lens of concepts specified by users. Existing
methods assume that the examples illustrating a concept are mapped in a fixed
direction of the DNN's latent space. When this holds true, the concept can be
represented by a concept activation vector (CAV) pointing in that direction. In
this work, we propose to relax this assumption by allowing concept examples to
be scattered across different clusters in the DNN's latent space. Each concept
is then represented by a region of the DNN's latent space that includes these
clusters and that we call concept activation region (CAR). To formalize this
idea, we introduce an extension of the CAV formalism that is based on the
kernel trick and support vector classifiers. This CAR formalism yields global
concept-based explanations and local concept-based feature importance. We prove
that CAR explanations built with radial kernels are invariant under latent
space isometries. In this way, CAR assigns the same explanations to latent
spaces that have the same geometry. We further demonstrate empirically that
CARs offer (1) more accurate descriptions of how concepts are scattered in the
DNN's latent space; (2) global explanations that are closer to human concept
annotations and (3) concept-based feature importance that meaningfully relate
concepts with each other. Finally, we use CARs to show that DNNs can
autonomously rediscover known scientific concepts, such as the prostate cancer
grading system.

本文提出一种新的概念激活区（CAR）的建立方式，通过采用核技巧和支持向量分类器来实现 CAR 的建立，以实现全局的概念解释和局部的概念特征重要性，经实证证明 CAR 可以更准确地描述 DNN 的潜在空间中的概念分布，并且可以用于 DNN 自行学习已知的科学概念，例如前列腺癌分级系统