We address the challenge of acquiring real-world manipulation skills with a
scalable framework.Inspired by the success of large-scale auto-regressive
prediction in Large Language Models (LLMs), we hold the belief that identifying
an appropriate prediction target capable of leveraging large-scale datasets is
crucial for achieving efficient and universal learning. Therefore, we propose
to utilize flow, which represents the future trajectories of 3D points on
objects of interest, as an ideal prediction target in robot learning. To
exploit scalable data resources, we turn our attention to cross-embodiment
datasets. We develop, for the first time, a language-conditioned prediction
model directly from large-scale RGBD human video datasets. Our predicted flow
offers actionable geometric and physics guidance, thus facilitating stable
zero-shot skill transfer in real-world scenarios.We deploy our method with a
policy based on closed-loop flow prediction. Remarkably, without any additional
training, our method achieves an impressive 81% success rate in human-to-robot
skill transfer, covering 18 tasks in 6 scenes. Our framework features the
following benefits: (1) scalability: leveraging cross-embodiment data
resources; (2) universality: multiple object categories, including rigid,
articulated, and soft bodies; (3) stable skill transfer: providing actionable
guidance with a small inference domain-gap. These lead to a new pathway towards
scalable general robot learning. Data, code, and model weights will be made
publicly available.

提出了一种基于预测流的机器人学习方法，能够实现在现实场景中稳定且高效的技能转移，涵盖了现实环境中的多个目标类别，通过利用大规模数据集实现了可扩展的普适机器人学习。