We study the problem of learning a robot policy to follow natural language
instructions that can be easily extended to reason about new objects. We
introduce a few-shot language-conditioned object grounding method trained from
augmented reality data that uses exemplars to identify objects and align them
to their mentions in instructions. We present a learned map representation that
encodes object locations and their instructed use, and construct it from our
few-shot grounding output. We integrate this mapping approach into an
instruction-following policy, thereby allowing it to reason about previously
unseen objects at test-time by simply adding exemplars. We evaluate on the task
of learning to map raw observations and instructions to continuous control of a
physical quadcopter. Our approach significantly outperforms the prior state of
the art in the presence of new objects, even when the prior approach observes
all objects during training.

该研究探讨了如何通过学习一个机器人策略来遵循自然语言指令，从而轻松地将其扩展以推理新对象，并介绍了一个基于数据增强的少量学习语言条件对象基础方法，以识别对象并将其与指令中的对象匹配。然后，将其整合到遵循指令的策略中，从而使其能够通过添加样例来推理出先前未见过的对象。经过评估，该方法在测试时比现有技术水平表现得更好，即使先前的方法在训练过程中观察到所有的对象。