Behavioral skills or policies for autonomous agents are conventionally
learned from reward functions, via reinforcement learning, or from
demonstrations, via imitation learning. However, both modes of task
specification have their disadvantages: reward functions require manual
engineering, while demonstrations require a human expert to be able to actually
perform the task in order to generate the demonstration. Instruction following
from natural language instructions provides an appealing alternative: in the
same way that we can specify goals to other humans simply by speaking or
writing, we would like to be able to specify tasks for our machines. However, a
single instruction may be insufficient to fully communicate our intent or, even
if it is, may be insufficient for an autonomous agent to actually understand
how to perform the desired task. In this work, we propose an interactive
formulation of the task specification problem, where iterative language
corrections are provided to an autonomous agent, guiding it in acquiring the
desired skill. Our proposed language-guided policy learning algorithm can
integrate an instruction and a sequence of corrections to acquire new skills
very quickly. In our experiments, we show that this method can enable a policy
to follow instructions and corrections for simulated navigation and
manipulation tasks, substantially outperforming direct, non-interactive
instruction following.

本文提出了一种交互式的任务规范问题形式，通过自然语言指令和一系列语言修正来引导自主代理获得新技能，实现快速获取新技能，显著优于直接非交互式指令跟随。