Explanatory Interactive Learning (XIL) collects user feedback on visual model
explanations to implement a Human-in-the-Loop (HITL) based interactive learning
scenario. Different user feedback types will have different impacts on user
experience and the cost associated with collecting feedback since different
feedback types involve different levels of image annotation. Although XIL has
been used to improve classification performance in multiple domains, the impact
of different user feedback types on model performance and explanation accuracy
is not well studied. To guide future XIL work we compare the effectiveness of
two different user feedback types in image classification tasks: (1)
instructing an algorithm to ignore certain spurious image features, and (2)
instructing an algorithm to focus on certain valid image features. We use
explanations from a Gradient-weighted Class Activation Mapping (GradCAM) based
XIL model to support both feedback types. We show that identifying and
annotating spurious image features that a model finds salient results in
superior classification and explanation accuracy than user feedback that tells
a model to focus on valid image features.

本研究使用了 Explanatory Interactive Learning 方法中的 Gradient-weighted Class Activation Mapping 模型，比较了两种用户反馈类型对图像分类任务的影响，结果表明，指出并注释模型找到的虚假图像特征比告诉模型专注于有效图像特征的反馈会导致更好的分类和解释准确性。

反馈类型对解释性互动学习的影响

Impact of Feedback Type on Explanatory Interactive Learning

Most explanation methods in deep learning map importance estimates for a
model's prediction back to the original input space. These "visual"
explanations are often insufficient, as the model's actual concept remains
elusive. Moreover, without insights into the model's semantic concept, it is
difficult -- if not impossible -- to intervene on the model's behavior via its
explanations, called Explanatory Interactive Learning. Consequently, we propose
to intervene on a Neuro-Symbolic scene representation, which allows one to
revise the model on the semantic level, e.g. "never focus on the color to make
your decision". We compiled a novel confounded visual scene data set, the
CLEVR-Hans data set, capturing complex compositions of different objects. The
results of our experiments on CLEVR-Hans demonstrate that our semantic
explanations, i.e. compositional explanations at a per-object level, can
identify confounders that are not identifiable using "visual" explanations
only. More importantly, feedback on this semantic level makes it possible to
revise the model from focusing on these factors.

该研究提出了一种基于神经符号场景表征的解释交互学习方法，它能够在语义层面上对模型进行干预，通过对物体的组合性解释来获得更好的效果。

正确的概念：通过与其解释交互来修订神经符号概念

Right for the Right Concept: Revising Neuro-Symbolic Concepts by  Interacting with their Explanations

Deep neural networks have shown excellent performances in many real-world
applications. Unfortunately, they may show "Clever Hans"-like behavior---making
use of confounding factors within datasets---to achieve high performance. In
this work, we introduce the novel learning setting of "explanatory interactive
learning" (XIL) and illustrate its benefits on a plant phenotyping research
task. XIL adds the scientist into the training loop such that she interactively
revises the original model via providing feedback on its explanations. Our
experimental results demonstrate that XIL can help avoiding Clever Hans moments
in machine learning and encourages (or discourages, if appropriate) trust into
the underlying model.

该研究介绍了 “explanatory interactive learning”（XIL）的新型学习模式，并通过植物表型研究任务展示了它的好处，结果表明 XIL 可以帮助避免机器学习中的 “Clever Hans” 行为，鼓励或不鼓励对基础模型的信任。