Text-based games are long puzzles or quests, characterized by a sequence of
sparse and potentially deceptive rewards. They provide an ideal platform to
develop agents that perceive and act upon the world using a combinatorially
sized natural language state-action space. Standard Reinforcement Learning
agents are poorly equipped to effectively explore such spaces and often
struggle to overcome bottlenecks---states that agents are unable to pass
through simply because they do not see the right action sequence enough times
to be sufficiently reinforced. We introduce Q*BERT, an agent that learns to
build a knowledge graph of the world by answering questions, which leads to
greater sample efficiency. To overcome bottlenecks, we further introduce
MC!Q*BERT an agent that uses an knowledge-graph-based intrinsic motivation to
detect bottlenecks and a novel exploration strategy to efficiently learn a
chain of policy modules to overcome them. We present an ablation study and
results demonstrating how our method outperforms the current state-of-the-art
on nine text games, including the popular game, Zork, where, for the first
time, a learning agent gets past the bottleneck where the player is eaten by a
Grue.

我们介绍了 Q*Bert 和 MC!Q*BERT 两个代理，在自然语言状态 - 动作空间中通过回答问题构建知识图谱来提高样本效率，并通过探索策略和内在动机检测瓶颈来有效探索空间，在九个文本游戏中超越了当前最先进的方法，包括 Zork 游戏。