We explore using latent natural language instructions as an expressive and
compositional representation of complex actions for hierarchical decision
making. Rather than directly selecting micro-actions, our agent first generates
a latent plan in natural language, which is then executed by a separate model.
We introduce a challenging real-time strategy game environment in which the
actions of a large number of units must be coordinated across long time scales.
We gather a dataset of 76 thousand pairs of instructions and executions from
human play, and train instructor and executor models. Experiments show that
models using natural language as a latent variable significantly outperform
models that directly imitate human actions. The compositional structure of
language proves crucial to its effectiveness for action representation. We also
release our code, models and data.

使用自然语言隐式指令作为复杂动作的表达和组合，而不是直接选择微操作，本文基于 76,000 个自然语言指令和操作的数据集在实时策略游戏环境中训练模型，结果表明使用自然语言作为潜变量的模型在动作表达中的有效性显著优于直接模仿人类动作的模型，并且语言的组合结构对其有效性具有关键作用。同时提供了代码、模型和数据。

通过生成和遵循自然语言指令进行分层决策

Hierarchical Decision Making by Generating and Following Natural  Language Instructions

The power grid is a complex and vital system that necessitates careful
reliability management. Managing the grid is a difficult problem with multiple
time scales of decision making and stochastic behavior due to renewable energy
generations, variable demand and unplanned outages. Solving this problem in the
face of uncertainty requires a new methodology with tractable algorithms. In
this work, we introduce a new model for hierarchical decision making in complex
systems. We apply reinforcement learning (RL) methods to learn a proxy, i.e., a
level of abstraction, for real-time power grid reliability. We devise an
algorithm that alternates between slow time-scale policy improvement, and fast
time-scale value function approximation. We compare our results to prevailing
heuristics, and show the strength of our method.

本文实现了一种具有层次决策制定的模型用于电网可靠性管理，使用强化学习算法学习实时电网可靠性的抽象，该算法交替进行快慢时间尺度的价值函数逼近和策略改进，并与先前的启发式算法进行比较，结果表明本方法的优越性。