We present an object-centric approach to empower robots to learn vision-based
manipulation skills from human videos. We investigate the problem of imitating
robot manipulation from a single human video in the open-world setting, where a
robot must learn to manipulate novel objects from one video demonstration. We
introduce ORION, an algorithm that tackles the problem by extracting an
object-centric manipulation plan from a single RGB-D video and deriving a
policy that conditions on the extracted plan. Our method enables the robot to
learn from videos captured by daily mobile devices such as an iPad and
generalize the policies to deployment environments with varying visual
backgrounds, camera angles, spatial layouts, and novel object instances. We
systematically evaluate our method on both short-horizon and long-horizon
tasks, demonstrating the efficacy of ORION in learning from a single human
video in the open world. Videos can be found in the project website
this https URL

通过从单个人类视频中提取以对象为中心的操纵计划并推导出条件为提取计划的策略，我们提出了一种以对象为中心的方法，为机器人学习基于视觉的操纵技能提供支持。我们的方法允许机器人从 iPad 等日常移动设备捕获的视频中学习，并将策略推广到具有不同视觉背景、摄像机角度、空间布局和新颖对象实例的部署环境中。通过对短程和长程任务进行系统评估，我们展示了 ORION 在从单个人类视频中学习开放世界中的有效性。