Collaborative tasks often begin with partial task knowledge and incomplete
initial plans from each partner. To complete these tasks, agents need to engage
in situated communication with their partners and coordinate their partial
plans towards a complete plan to achieve a joint task goal. While such
collaboration seems effortless in a human-human team, it is highly challenging
for human-AI collaboration. To address this limitation, this paper takes a step
towards collaborative plan acquisition, where humans and agents strive to learn
and communicate with each other to acquire a complete plan for joint tasks.
Specifically, we formulate a novel problem for agents to predict the missing
task knowledge for themselves and for their partners based on rich perceptual
and dialogue history. We extend a situated dialogue benchmark for symmetric
collaborative tasks in a 3D blocks world and investigate computational
strategies for plan acquisition. Our empirical results suggest that predicting
the partner's missing knowledge is a more viable approach than predicting one's
own. We show that explicit modeling of the partner's dialogue moves and mental
states produces improved and more stable results than without. These results
provide insight for future AI agents that can predict what knowledge their
partner is missing and, therefore, can proactively communicate such information
to help their partner acquire such missing knowledge toward a common
understanding of joint tasks.

为了解决 AI 与人类之间协作中的问题，本文提出了一种新颖的计算问题，即为智能体预测它们自己和它们的合作者缺失的任务知识，通过丰富的感知和对话历史进行沟通和学习，以实现联合任务的完成，结果表明，预测合作者缺失知识是一种更可行的方法，且明确建模合作者对话移动和心理状态可以提供更好的结果。

基于心智模型的情境对话中协同计划获取

Towards Collaborative Plan Acquisition through Theory of Mind Modeling  in Situated Dialogue

While it is known that communication facilitates cooperation in multi-agent
settings, it is unclear how to design artificial agents that can learn to
effectively and efficiently communicate with each other. Much research on
communication emergence uses reinforcement learning (RL) and explores
unsituated communication in one-step referential tasks -- the tasks are not
temporally interactive and lack time pressures typically present in natural
communication. In these settings, agents may successfully learn to communicate,
but they do not learn to exchange information concisely -- they tend towards
over-communication and an inefficient encoding. Here, we explore situated
communication in a multi-step task, where the acting agent has to forgo an
environmental action to communicate. Thus, we impose an opportunity cost on
communication and mimic the real-world pressure of passing time. We compare
communication emergence under this pressure against learning to communicate
with a cost on articulation effort, implemented as a per-message penalty (fixed
and progressively increasing). We find that while all tested pressures can
disincentivise over-communication, situated communication does it most
effectively and, unlike the cost on effort, does not negatively impact
emergence. Implementing an opportunity cost on communication in a temporally
extended environment is a step towards embodiment, and might be a pre-condition
for incentivising efficient, human-like communication.

本研究使用强化学习来探索多智能体间的通信，模拟真实世界的压力，并比较实施在多步骤任务中的自然通信和基于非自然通信的良好交流之间的差异，研究结果表明，自然通信可以更有效地激励有效，类似于人类的沟通。

不再过度沟通：基于情境的 RL 代理学习简明的通信协议

Over-communicate no more: Situated RL agents learn concise communication protocols

In the real world, autonomous driving agents navigate in highly dynamic
environments full of unexpected situations where pre-trained models are
unreliable. In these situations, what is immediately available to vehicles is
often only human operators. Empowering autonomous driving agents with the
ability to navigate in a continuous and dynamic environment and to communicate
with humans through sensorimotor-grounded dialogue becomes critical. To this
end, we introduce Dialogue On the ROad To Handle Irregular Events (DOROTHIE), a
novel interactive simulation platform that enables the creation of unexpected
situations on the fly to support empirical studies on situated communication
with autonomous driving agents. Based on this platform, we created the Situated
Dialogue Navigation (SDN), a navigation benchmark of 183 trials with a total of
8415 utterances, around 18.7 hours of control streams, and 2.9 hours of trimmed
audio. SDN is developed to evaluate the agent's ability to predict dialogue
moves from humans as well as generate its own dialogue moves and physical
navigation actions. We further developed a transformer-based baseline model for
these SDN tasks. Our empirical results indicate that language guided-navigation
in a highly dynamic environment is an extremely difficult task for end-to-end
models. These results will provide insight towards future work on robust
autonomous driving agents. The DOROTHIE platform, SDN benchmark, and code for
the baseline model are available at this https URL.

介绍了一种基于 DOROTHIE 平台的 Situated Dialogue Navigation (SDN) 导航基准，考察自主驾驶代理通过对话进行动态环境下导航的能力，结果表明，端到端模型在这种高度动态的语言引导导航任务中是非常困难的。