Complex, multi-task problems have proven to be difficult to solve efficiently
in a sparse-reward reinforcement learning setting. In order to be sample
efficient, multi-task learning requires reuse and sharing of low-level
policies. To facilitate the automatic decomposition of hierarchical tasks, we
propose the use of step-by-step human demonstrations in the form of natural
language instructions and action trajectories. We introduce a dataset of such
demonstrations in a crafting-based grid world. Our model consists of a
high-level language generator and low-level policy, conditioned on language. We
find that human demonstrations help solve the most complex tasks. We also find
that incorporating natural language allows the model to generalize to unseen
tasks in a zero-shot setting and to learn quickly from a few demonstrations.
Generalization is not only reflected in the actions of the agent, but also in
the generated natural language instructions in unseen tasks. Our approach also
gives our trained agent interpretable behaviors because it is able to generate
a sequence of high-level descriptions of its actions.

本文提出通过使用自然语言指令和行动轨迹演示来自动分解分层任务，以解决在稀疏奖励的强化学习设置中的复杂多任务问题，并证明人类演示有助于解决最复杂的任务，同时允许该模型再未见数据的情况下推广学习，从而使训练好的代理人具有可解释的行为。