Concept Bottleneck Models (CBMs) first map raw input(s) to a vector of
human-defined concepts, before using this vector to predict a final
classification. We might therefore expect CBMs capable of predicting concepts
based on distinct regions of an input. In doing so, this would support human
interpretation when generating explanations of the model's outputs to visualise
input features corresponding to concepts. The contribution of this paper is
threefold: Firstly, we expand on existing literature by looking at relevance
both from the input to the concept vector, confirming that relevance is
distributed among the input features, and from the concept vector to the final
classification where, for the most part, the final classification is made using
concepts predicted as present. Secondly, we report a quantitative evaluation to
measure the distance between the maximum input feature relevance and the ground
truth location; we perform this with the techniques, Layer-wise Relevance
Propagation (LRP), Integrated Gradients (IG) and a baseline gradient approach,
finding LRP has a lower average distance than IG. Thirdly, we propose using the
proportion of relevance as a measurement for explaining concept importance.

本文研究对于 Concept Bottleneck Models（CBMs）中的输入特征与概念向量之间的关联性，以及如何使用 Layer-wise Relevance Propagation（LRP）、Integrated Gradients（IG）等方法来解释 CBMs 的输出结果，提出了用权重比例来衡量概念重要性的新方法。