Text-based games present a unique challenge for autonomous agents to operate in natural language and handle enormous action spaces. In this paper, we propose the Contextual Action Language Model (CALM) to generate a compact set of action candidates at each game state. Our key insight is to train language models on human gameplay, where people demonstrate linguistic priors and a general game sense for promising actions conditioned on game history. We combine CALM with a reinforcement learning agent which re-ranks the generated action candidates to maximize in-game rewards. We evaluate our approach using the Jericho benchmark, on games unseen by CALM during training. Our method obtains a 69% relative improvement in average game score over the previous state-of-the-art model. Surprisingly, on half of these games, CALM is competitive with or better than other models that have access to ground truth admissible actions. Code and data are available at https://github.com/princeton-nlp/calm-textgame.

本文提出了上下文行动语言模型(CALM)，该模型结合人类玩家的语言先验以及游戏历史信息生成紧凑的候选操作列表，并结合强化学习代理对其进行排序以最大化游戏收益，我们的实验使用Jericho基准测试游戏并在训练期间未见过的游戏中获得了69％的相对平均游戏得分改进。

保持冷静探索：基于语言模型的基于文本的游戏行动生成