Compositional Explanations is a method for identifying logical formulas of
concepts that approximate the neurons' behavior. However, these explanations
are linked to the small spectrum of neuron activations (i.e., the highest ones)
used to check the alignment, thus lacking completeness. In this paper, we
propose a generalization, called Clustered Compositional Explanations, that
combines Compositional Explanations with clustering and a novel search
heuristic to approximate a broader spectrum of the neurons' behavior. We define
and address the problems connected to the application of these methods to
multiple ranges of activations, analyze the insights retrievable by using our
algorithm, and propose desiderata qualities that can be used to study the
explanations returned by different algorithms.

本文提出了一种称为集群组合解释的概括方法，它结合了组合解释、聚类和新颖的搜索启发式，以近似更广范围的神经元行为。我们定义并解决了应用这些方法到多个激活范围的问题，并分析了通过使用我们的算法可获取的洞见，并提出了可以用来研究不同算法返回的解释的期望品质。

以聚类组成性解释增进对神经元的全面理解

Towards a fuller understanding of neurons with Clustered Compositional  Explanations

Most explanation methods in deep learning map importance estimates for a
model's prediction back to the original input space. These "visual"
explanations are often insufficient, as the model's actual concept remains
elusive. Moreover, without insights into the model's semantic concept, it is
difficult -- if not impossible -- to intervene on the model's behavior via its
explanations, called Explanatory Interactive Learning. Consequently, we propose
to intervene on a Neuro-Symbolic scene representation, which allows one to
revise the model on the semantic level, e.g. "never focus on the color to make
your decision". We compiled a novel confounded visual scene data set, the
CLEVR-Hans data set, capturing complex compositions of different objects. The
results of our experiments on CLEVR-Hans demonstrate that our semantic
explanations, i.e. compositional explanations at a per-object level, can
identify confounders that are not identifiable using "visual" explanations
only. More importantly, feedback on this semantic level makes it possible to
revise the model from focusing on these factors.

该研究提出了一种基于神经符号场景表征的解释交互学习方法，它能够在语义层面上对模型进行干预，通过对物体的组合性解释来获得更好的效果。