Intelligent agents such as robots are increasingly deployed in real-world,
safety-critical settings. It is vital that these agents are able to explain the
reasoning behind their decisions to human counterparts; however, their behavior
is often produced by uninterpretable models such as deep neural networks. We
propose an approach to generate natural language explanations for an agent's
behavior based only on observations of states and actions, thus making our
method independent from the underlying model's representation. For such models,
we first learn a behavior representation and subsequently use it to produce
plausible explanations with minimal hallucination while affording user
interaction with a pre-trained large language model. We evaluate our method in
a multi-agent search-and-rescue environment and demonstrate the effectiveness
of our explanations for agents executing various behaviors. Through user
studies and empirical experiments, we show that our approach generates
explanations as helpful as those produced by a human domain expert while
enabling beneficial interactions such as clarification and counterfactual
queries.

基于状态和行为观察生成自然语言解释，不依赖于基础模型的表示，能解释智能代理行为，使用户能够与预训练的大型语言模型进行交互并生成有助于解释与推理的解释。

理解代理程序：利用大型语言模型进行行为解释

Understanding Your Agent: Leveraging Large Language Models for Behavior  Explanation

Decision analysis deals with modeling and enhancing decision processes. A
principal challenge in improving behavior is in obtaining a transparent
description of existing behavior in the first place. In this paper, we develop
an expressive, unifying perspective on inverse decision modeling: a framework
for learning parameterized representations of sequential decision behavior.
First, we formalize the forward problem (as a normative standard), subsuming
common classes of control behavior. Second, we use this to formalize the
inverse problem (as a descriptive model), generalizing existing work on
imitation/reward learning -- while opening up a much broader class of research
problems in behavior representation. Finally, we instantiate this approach with
an example (inverse bounded rational control), illustrating how this structure
enables learning (interpretable) representations of (bounded) rationality --
while naturally capturing intuitive notions of suboptimal actions, biased
beliefs, and imperfect knowledge of environments.

通过开发一个逆向决策建模的框架，我们在这篇论文中提出了一种表达顺序决策行为的参数化表示的方法，该方法不仅泛化了现有的模仿 / 奖励学习工作，而且还打开了更广泛的行为表征研究问题的研究领域。