To substantially enhance robot intelligence, there is a pressing need to
develop a large model that enables general-purpose robots to proficiently
undertake a broad spectrum of manipulation tasks, akin to the versatile
task-planning ability exhibited by LLMs. The vast diversity in objects, robots,
and manipulation tasks presents huge challenges. Our work introduces a
comprehensive framework to develop a foundation model for general robotic
manipulation that formalizes a manipulation task as contact synthesis.
Specifically, our model takes as input object and robot manipulator point
clouds, object physical attributes, target motions, and manipulation region
masks. It outputs contact points on the object and associated contact forces or
post-contact motions for robots to achieve the desired manipulation task. We
perform extensive experiments both in the simulation and real-world settings,
manipulating articulated rigid objects, rigid objects, and deformable objects
that vary in dimensionality, ranging from one-dimensional objects like ropes to
two-dimensional objects like cloth and extending to three-dimensional objects
such as plasticine. Our model achieves average success rates of around 90\%.
Supplementary materials and videos are available on our project website at
this https URL

为了极大增强机器人智能，我们提出了一个全面的框架来开发通用机器人操作的基础模型，并通过形式化操作任务为接触综合来实现。我们的模型以物体和机器人的点云、物体的物理属性、目标运动和操作区域掩码作为输入，并输出物体上的接触点以及机器人实现所需的接触力或接触后的运动。在仿真和实际世界环境中进行了广泛实验，操作的对象包括关节刚体、刚体和可变形物体，其维度范围从一维物体如绳索到二维物体如布料以及三维物体如软陶。我们的模型达到了约 90% 的平均成功率。