Neuro-symbolic reinforcement learning (NS-RL) has emerged as a promising
paradigm for explainable decision-making, characterized by the interpretability
of symbolic policies. For tasks with visual observations, NS-RL entails
structured representations for states, but previous algorithms are unable to
refine the structured states with reward signals due to a lack of efficiency.
Accessibility is also an issue, as extensive domain knowledge is required to
interpret current symbolic policies. In this paper, we present a framework that
is capable of learning structured states and symbolic policies simultaneously,
whose key idea is to overcome the efficiency bottleneck by distilling vision
foundation models into a scalable perception module. Moreover, we design a
pipeline that uses large language models to generate concise and readable
language explanations for policies and decisions. In experiments on nine Atari
tasks, our approach demonstrates substantial performance gains over existing
NSRL methods. We also showcase explanations for policies and decisions.

该研究介绍了一种神经符号强化学习（Neuro-symbolic reinforcement learning，NS-RL）的框架，通过将视觉基础模型精简为可扩展的感知模块，可以同时学习结构化状态和符号策略，并使用大型语言模型生成简明易读的策略和决策解释，实验结果显示在九个 Atari 任务中，该方法相比现有的 NS-RL 方法获得显著的性能提升，并展示了策略和决策的解释。

全面神经符号视觉强化学习与语言解释

INSIGHT: End-to-End Neuro-Symbolic Visual Reinforcement Learning with  Language Explanations

Advances in multi-agent reinforcement learning (MARL) enable sequential
decision making for a range of exciting multi-agent applications such as
cooperative AI and autonomous driving. Explaining agent decisions is crucial
for improving system transparency, increasing user satisfaction, and
facilitating human-agent collaboration. However, existing works on explainable
reinforcement learning mostly focus on the single-agent setting and are not
suitable for addressing challenges posed by multi-agent environments. We
present novel methods to generate two types of policy explanations for MARL:
(i) policy summarization about the agent cooperation and task sequence, and
(ii) language explanations to answer queries about agent behavior. Experimental
results on three MARL domains demonstrate the scalability of our methods. A
user study shows that the generated explanations significantly improve user
performance and increase subjective ratings on metrics such as user
satisfaction.

本文介绍了针对多智能体强化学习提出两种方法生成策略解释的手段：关于智能体协作和任务序列的策略总结和回答智能体行为问题的语言解释。在三个多智能体应用领域的实验结果及用户研究中，表明了该方法的可扩展性和显著提高了使用者满意度和性能评价结果。

多智能体强化学习的策略解释

Toward Policy Explanations for Multi-Agent Reinforcement Learning

We present a novel large-scale dataset and accompanying machine learning
models aimed at providing a detailed understanding of the interplay between
visual content, its emotional effect, and explanations for the latter in
language. In contrast to most existing annotation datasets in computer vision,
we focus on the affective experience triggered by visual artworks and ask the
annotators to indicate the dominant emotion they feel for a given image and,
crucially, to also provide a grounded verbal explanation for their emotion
choice. As we demonstrate below, this leads to a rich set of signals for both
the objective content and the affective impact of an image, creating
associations with abstract concepts (e.g., "freedom" or "love"), or references
that go beyond what is directly visible, including visual similes and
metaphors, or subjective references to personal experiences. We focus on visual
art (e.g., paintings, artistic photographs) as it is a prime example of imagery
created to elicit emotional responses from its viewers. Our dataset, termed
ArtEmis, contains 439K emotion attributions and explanations from humans, on
81K artworks from WikiArt. Building on this data, we train and demonstrate a
series of captioning systems capable of expressing and explaining emotions from
visual stimuli. Remarkably, the captions produced by these systems often
succeed in reflecting the semantic and abstract content of the image, going
well beyond systems trained on existing datasets. The collected dataset and
developed methods are available at this https URL

论文提供了一个大型情感分析数据集 (ArtEmis)，用于分析视觉艺术作品与情感体验之间的关联，旨在通过机器学习模型提供情感影响和语言解释之间的详细理解。通过调查人类对于 81,000 幅维基百科艺术作品的情感归属权和解释等信息，对与视觉作品产生情感联系的数据信号进行了精细的建模，进而提出了能够清晰表达视觉刺激的情感并解释其语义和抽象内容的系统。