We explore using latent natural language instructions as an expressive and
compositional representation of complex actions for hierarchical decision
making. Rather than directly selecting micro-actions, our agent first generates
a latent plan in natural language, which is then executed by a separate model.
We introduce a challenging real-time strategy game environment in which the
actions of a large number of units must be coordinated across long time scales.
We gather a dataset of 76 thousand pairs of instructions and executions from
human play, and train instructor and executor models. Experiments show that
models using natural language as a latent variable significantly outperform
models that directly imitate human actions. The compositional structure of
language proves crucial to its effectiveness for action representation. We also
release our code, models and data.

使用自然语言隐式指令作为复杂动作的表达和组合，而不是直接选择微操作，本文基于 76,000 个自然语言指令和操作的数据集在实时策略游戏环境中训练模型，结果表明使用自然语言作为潜变量的模型在动作表达中的有效性显著优于直接模仿人类动作的模型，并且语言的组合结构对其有效性具有关键作用。同时提供了代码、模型和数据。