World models improve a learning agent's ability to efficiently operate in
interactive and situated environments. This work focuses on the task of
building world models of text-based game environments. Text-based games, or
interactive narratives, are reinforcement learning environments in which agents
perceive and interact with the world using textual natural language. These
environments contain long, multi-step puzzles or quests woven through a world
that is filled with hundreds of characters, locations, and objects. Our world
model learns to simultaneously: (1) predict changes in the world caused by an
agent's actions when representing the world as a knowledge graph; and (2)
generate the set of contextually relevant natural language actions required to
operate in the world. We frame this task as a Set of Sequences generation
problem by exploiting the inherent structure of knowledge graphs and actions
and introduce both a transformer-based multi-task architecture and a loss
function to train it. A zero-shot ablation study on never-before-seen textual
worlds shows that our methodology significantly outperforms existing textual
world modeling techniques as well as the importance of each of our
contributions.

该论文聚焦于构建基于文本的游戏环境的世界模型，利用知识图谱和自然语言行动生成模型，提升增强学习智能体在该环境下的效率。通过零样本消融实验，表明该方法显著优于现有的文本世界建模技术及其各自贡献的重要性。

学习基于知识图谱的文本环境世界模型

Learning Knowledge Graph-based World Models of Textual Environments

Response generation for task-oriented dialogues implicitly optimizes two
objectives at the same time: task completion and language quality. Conditioned
response generation serves as an effective approach to separately and better
optimize these two objectives. Such an approach relies on system action
annotations which are expensive to obtain. To alleviate the need of action
annotations, latent action learning is introduced to map each utterance to a
latent representation. However, this approach is prone to over-dependence on
the training data, and the generalization capability is thus restricted. To
address this issue, we propose to learn natural language actions that represent
utterances as a span of words. This explicit action representation promotes
generalization via the compositional structure of language. It also enables an
explainable generation process. Our proposed unsupervised approach learns a
memory component to summarize system utterances into a short span of words. To
further promote a compact action representation, we propose an auxiliary task
that restores state annotations as the summarized dialogue context using the
memory component. Our proposed approach outperforms latent action baselines on
MultiWOZ, a benchmark multi-domain dataset.

本研究提出了一种无监督的方法，旨在将用户话语映射成一段词组的自然语言行为，从而优化任务完成和语言质量。该方法通过经验性学习得到一个能够将系统话语总结成一个简短词组的记忆组件，以促进紧凑的行为表示，并进一步通过辅助任务优化智能对话系统的自动化程度。