We present Neural Descriptor Fields (NDFs), an object representation that
encodes both points and relative poses between an object and a target (such as
a robot gripper or a rack used for hanging) via category-level descriptors. We
employ this representation for object manipulation, where given a task
demonstration, we want to repeat the same task on a new object instance from
the same category. We propose to achieve this objective by searching (via
optimization) for the pose whose descriptor matches that observed in the
demonstration. NDFs are conveniently trained in a self-supervised fashion via a
3D auto-encoding task that does not rely on expert-labeled keypoints. Further,
NDFs are SE(3)-equivariant, guaranteeing performance that generalizes across
all possible 3D object translations and rotations. We demonstrate learning of
manipulation tasks from few (5-10) demonstrations both in simulation and on a
real robot. Our performance generalizes across both object instances and 6-DoF
object poses, and significantly outperforms a recent baseline that relies on 2D
descriptors. Project website: this https URL

提出了神经描述器场（NDF），一种通过类别级别描述符编码物体和目标（例如机器人夹子或挂架）之间的点和相对姿态的物体表示。通过自我监督学习 3D 自编码任务来方便地训练 NDFs，并将其用于从少量示范中学习物体操作任务，具有跨越所有可能的 3D 物体翻转和旋转的性能。