A generalist robot equipped with learned skills must be able to perform many
tasks in many different environments. However, zero-shot generalization to new
settings is not always possible. When the robot encounters a new environment or
object, it may need to finetune some of its previously learned skills to
accommodate this change. But crucially, previously learned behaviors and models
should still be suitable to accelerate this relearning. In this paper, we aim
to study how generative models of possible outcomes can allow a robot to learn
visual representations of affordances, so that the robot can sample potentially
possible outcomes in new situations, and then further train its policy to
achieve those outcomes. In effect, prior data is used to learn what kinds of
outcomes may be possible, such that when the robot encounters an unfamiliar
setting, it can sample potential outcomes from its model, attempt to reach
them, and thereby update both its skills and its outcome model. This approach,
visuomotor affordance learning (VAL), can be used to train goal-conditioned
policies that operate on raw image inputs, and can rapidly learn to manipulate
new objects via our proposed affordance-directed exploration scheme. We show
that VAL can utilize prior data to solve real-world tasks such drawer opening,
grasping, and placing objects in new scenes with only five minutes of online
experience in the new scene.

本文旨在研究如何通过生成模型来让一个通用机器人学习视觉契合性的表征，以便于机器人可以在新环境中使用它的模型进行采样，进而进一步训练其策略从而达成其目标。这种做法可以被用于训练以原始图像输入为操作对象的目标编码策略，并可以通过我们提出的契合性导向探索机制快速地学习如何操作新对象。我们展示，通过五分钟的在线学习就可以在新场景下使用之前的数据训练机器人完成抽屉打开，抓取和放置的任务。

通过想象视觉可供性学习新技能：我在这里能做什么？

What Can I Do Here? Learning New Skills by Imagining Visual Affordances

A generalist robot must be able to complete a variety of tasks in its
environment. One appealing way to specify each task is in terms of a goal
observation. However, learning goal-reaching policies with reinforcement
learning remains a challenging problem, particularly when hand-engineered
reward functions are not available. Learned dynamics models are a promising
approach for learning about the environment without rewards or task-directed
data, but planning to reach goals with such a model requires a notion of
functional similarity between observations and goal states. We present a
self-supervised method for model-based visual goal reaching, which uses both a
visual dynamics model as well as a dynamical distance function learned using
model-free reinforcement learning. Our approach learns entirely using offline,
unlabeled data, making it practical to scale to large and diverse datasets. In
our experiments, we find that our method can successfully learn models that
perform a variety of tasks at test-time, moving objects amid distractors with a
simulated robotic arm and even learning to open and close a drawer using a
real-world robot. In comparisons, we find that this approach substantially
outperforms both model-free and model-based prior methods. Videos and
visualizations are available here: this http URL

本文介绍了一种使用自监督学习方法和动力学模型和距离函数相结合的视觉目标到达方法，可用于训练通用机器人执行多种任务，该方法不需要手动设计奖励函数，仅使用无标签数据进行学习，并表现出较高的性能。