Installing probabilistic world models into artificial agents opens an
efficient channel for humans to communicate with and control these agents. In
addition to updating agent policies, humans can modify their internal world
models in order to influence their decisions. The challenge, however, is that
currently existing world models are difficult for humans to adapt because they
lack a natural communication interface. Aimed at addressing this shortcoming,
we develop Language-Guided World Models (LWMs), which can capture environment
dynamics by reading language descriptions. These models enhance agent
communication efficiency, allowing humans to simultaneously alter their
behavior on multiple tasks with concise language feedback. They also enable
agents to self-learn from texts originally written to instruct humans. To
facilitate the development of LWMs, we design a challenging benchmark based on
the game of MESSENGER (Hanjie et al., 2021), requiring compositional
generalization to new language descriptions and environment dynamics. Our
experiments reveal that the current state-of-the-art Transformer architecture
performs poorly on this benchmark, motivating us to design a more robust
architecture. To showcase the practicality of our proposed LWMs, we simulate a
scenario where these models augment the interpretability and safety of an agent
by enabling it to generate and discuss plans with a human before execution. By
effectively incorporating language feedback on the plan, the models boost the
agent performance in the real environment by up to three times without
collecting any interactive experiences in this environment.

安装概率世界模型到人工智能代理中，为人类与控制这些代理打开了一个高效的交流途径；我们开发了一种名为语言引导的世界模型（LWMs），通过阅读语言描述来捕捉环境动态，提高了代理的通信效率，同时允许人类用简洁的语言反馈在多个任务中同时改变行为。该研究证明了当前最先进的 Transformer 架构在该基准测试上表现不佳，激励我们设计更强大的架构。通过模拟展示了我们提出的 LWMs 的实用性，使代理能够在执行前生成和讨论计划，增强了代理的可解释性和安全性，并使其在真实环境中的性能提高了三倍，而无需在该环境中进行任何交互式经验的收集。