Most explanation methods in deep learning map importance estimates for a
model's prediction back to the original input space. These "visual"
explanations are often insufficient, as the model's actual concept remains
elusive. Moreover, without insights into the model's semantic concept, it is
difficult -- if not impossible -- to intervene on the model's behavior via its
explanations, called Explanatory Interactive Learning. Consequently, we propose
to intervene on a Neuro-Symbolic scene representation, which allows one to
revise the model on the semantic level, e.g. "never focus on the color to make
your decision". We compiled a novel confounded visual scene data set, the
CLEVR-Hans data set, capturing complex compositions of different objects. The
results of our experiments on CLEVR-Hans demonstrate that our semantic
explanations, i.e. compositional explanations at a per-object level, can
identify confounders that are not identifiable using "visual" explanations
only. More importantly, feedback on this semantic level makes it possible to
revise the model from focusing on these factors.

该研究提出了一种基于神经符号场景表征的解释交互学习方法，它能够在语义层面上对模型进行干预，通过对物体的组合性解释来获得更好的效果。

正确的概念：通过与其解释交互来修订神经符号概念

Right for the Right Concept: Revising Neuro-Symbolic Concepts by  Interacting with their Explanations

We propose a hierarchically structured reinforcement learning approach to
address the challenges of planning for generating coherent multi-sentence
stories for the visual storytelling task. Within our framework, the task of
generating a story given a sequence of images is divided across a two-level
hierarchical decoder. The high-level decoder constructs a plan by generating a
semantic concept (i.e., topic) for each image in sequence. The low-level
decoder generates a sentence for each image using a semantic compositional
network, which effectively grounds the sentence generation conditioned on the
topic. The two decoders are jointly trained end-to-end using reinforcement
learning. We evaluate our model on the visual storytelling (VIST) dataset.
Empirical results from both automatic and human evaluations demonstrate that
the proposed hierarchically structured reinforced training achieves
significantly better performance compared to a strong flat deep reinforcement
learning baseline.

我们提出了一种分层结构的强化学习方法，用于解决视觉叙事任务中生成连贯多句子故事的挑战，并在视觉叙事（VIST）数据集上评估了该模型的性能。