Instruction-following agents must ground language into their observation and
action spaces. Learning to ground language is challenging, typically requiring
domain-specific engineering or large quantities of human interaction data. To
address this challenge, we propose using pretrained vision-language models
(VLMs) to supervise embodied agents. We combine ideas from model distillation
and hindsight experience replay (HER), using a VLM to retroactively generate
language describing the agent's behavior. Simple prompting allows us to control
the supervision signal, teaching an agent to interact with novel objects based
on their names (e.g., planes) or their features (e.g., colors) in a 3D rendered
environment. Fewshot prompting lets us teach abstract category membership,
including pre-existing categories (food vs toys) and ad-hoc ones (arbitrary
preferences over objects). Our work outlines a new and effective way to use
internet-scale VLMs, repurposing the generic language grounding acquired by
such models to teach task-relevant groundings to embodied agents.

本文提出了一种利用预训练的视觉语言模型（VLMs）来监督具有目的地行动的体验型智能体从而让其学会与不同类别的对象交互的新型方式。通过少量提示和后见之明体验回放（HER）技术，我们可以使体验型智能体学习抽象类别的成员资格以及与特定任务相关的语境，在利用因特网规模的 VLMs 的通用语言基础的同时完成语言建模。