Interactive fiction games have emerged as an important application to improve
the generalization capabilities of language-based reinforcement learning (RL)
agents. Existing environments for interactive fiction games are domain-specific
or time-consuming to generate and do not train the RL agents to master a
specific set of skills. In this work, we introduce an interactive environment
for self-supervised RL, STARLING, for text-based games that bootstraps the
text-based RL agents with automatically generated games (based on the seed set
of game ideas) to boost the performance and generalization capabilities to
reach a goal of the target environment. These games let the agent hone their
skills on a predefined set of tasks. We create and test an environment with 100
games, generated using this automated framework that uses large language models
(GPT-3) and an interactive fiction game engine (based on Inform7) to provide
the user with the ability to generate more games under minimal human
supervision. Experimental results based on both the human participants and
baseline text-based RL agents reveal that current state-of-the-art text-based
RL agents cannot use previously learned skills in new situations at the level
humans can. These results enforce STARLING's potential to serve as a sandbox
environment for further research in self-supervised text-based RL.

自动化游戏生成的 STARLING 环境为基于文本的强化学习代理提供了提升性能和泛化能力的能力，以通过与预定义任务集上的训练来提高代理的技能水平。

STARLING：基于大型语言模型的文本强化学习自监督训练代理

STARLING: Self-supervised Training of Text-based Reinforcement Learning  Agent with Large Language Models

Interactive Fiction games are text-based simulations in which an agent
interacts with the world purely through natural language. They are ideal
environments for studying how to extend reinforcement learning agents to meet
the challenges of natural language understanding, partial observability, and
action generation in combinatorially-large text-based action spaces. We present
KG-A2C, an agent that builds a dynamic knowledge graph while exploring and
generates actions using a template-based action space. We contend that the dual
uses of the knowledge graph to reason about game state and to constrain natural
language generation are the keys to scalable exploration of combinatorially
large natural language actions. Results across a wide variety of IF games show
that KG-A2C outperforms current IF agents despite the exponential increase in
action space size.

本文介绍了 KG-A2C 代理，它通过构建动态知识图谱并使用基于模板的行动空间来生成动作，解决了在大规模组合自然语言行动空间中扩展强化学习代理以应对自然语言理解、部分可观察性和行动生成挑战的问题；从广泛的 IF 游戏结果来看，KG-A2C 优于当前的 IF 代理，尽管行动空间尺寸呈指数级增长。

自然语言动作空间的图形约束强化学习

Graph Constrained Reinforcement Learning for Natural Language Action  Spaces

A hallmark of human intelligence is the ability to understand and communicate
with language. Interactive Fiction games are fully text-based simulation
environments where a player issues text commands to effect change in the
environment and progress through the story. We argue that IF games are an
excellent testbed for studying language-based autonomous agents. In particular,
IF games combine challenges of combinatorial action spaces, language
understanding, and commonsense reasoning. To facilitate rapid development of
language-based agents, we introduce Jericho, a learning environment for
man-made IF games and conduct a comprehensive study of text-agents across a
rich set of games, highlighting directions in which agents can improve.

介绍了 Jericho，IF 游戏的学习环境，并通过对丰富游戏集合上的文本代理进行全面研究，强调了代理可以提高的方向。