Attribution methods are a popular class of explainability methods that use
heatmaps to depict the most important areas of an image that drive a model
decision. Nevertheless, recent work has shown that these methods have limited
utility in practice, presumably because they only highlight the most salient
parts of an image (i.e., 'where' the model looked) and do not communicate any
information about 'what' the model saw at those locations. In this work, we try
to fill in this gap with CRAFT -- a novel approach to identify both 'what' and
'where' by generating concept-based explanations. We introduce 3 new
ingredients to the automatic concept extraction literature: (i) a recursive
strategy to detect and decompose concepts across layers, (ii) a novel method
for a more faithful estimation of concept importance using Sobol indices, and
(iii) the use of implicit differentiation to unlock Concept Attribution Maps.
We conduct both human and computer vision experiments to demonstrate the
benefits of the proposed approach. We show that our recursive decomposition
generates meaningful and accurate concepts and that the proposed concept
importance estimation technique is more faithful to the model than previous
methods. When evaluating the usefulness of the method for human experimenters
on a human-defined utility benchmark, we find that our approach significantly
improves on two of the three test scenarios (while none of the current methods
including ours help on the third). Overall, our study suggests that, while much
work remains toward the development of general explainability methods that are
useful in practical scenarios, the identification of meaningful concepts at the
proper level of granularity yields useful and complementary information beyond
that afforded by attribution methods.

本研究提出了一种名为 CRAFT 的新方法，用于识别基于概念的解释，并介绍了 3 种新的自动概念提取方法。该方法的实验表明，适当粒度的有意义概念的识别可以提供有用的信息，并且比仅强调最显著部分的归因方法的效果好。