Dexterous manipulation with a multi-finger hand is one of the most
challenging problems in robotics. While recent progress in imitation learning
has largely improved the sample efficiency compared to Reinforcement Learning,
the learned policy can hardly generalize to manipulate novel objects, given
limited expert demonstrations. In this paper, we propose to learn dexterous
manipulation using large-scale demonstrations with diverse 3D objects in a
category, which are generated from a human grasp affordance model. This
generalizes the policy to novel object instances within the same category. To
train the policy, we propose a novel imitation learning objective jointly with
a geometric representation learning objective using our demonstrations. By
experimenting with relocating diverse objects in simulation, we show that our
approach outperforms baselines with a large margin when manipulating novel
objects. We also ablate the importance on 3D object representation learning for
manipulation. We include videos, code, and additional information on the
project website - this https URL .

本研究提出了使用大规模演示学习熟练操作多指手来处理机器人学中最具挑战性的问题。 主要采用人类抓握可行性模型生成 3D 物体演示，以实现策略的泛化。同时，本研究使用了新颖的模仿学习目标，以及几何表示学习目标，通过在仿真系统中重新定位多样的对象以证明本方法的有效性。

从人类抓握机会学习具有普适性的熟练操作

Learning Generalizable Dexterous Manipulation from Human Grasp  Affordance

While predicting robot grasps with parallel jaw grippers have been well
studied and widely applied in robot manipulation tasks, the study on natural
human grasp generation with a multi-finger hand remains a very challenging
problem. In this paper, we propose to generate human grasps given a 3D object
in the world. Our key observation is that it is crucial to model the
consistency between the hand contact points and object contact regions. That
is, we encourage the prior hand contact points to be close to the object
surface and the object common contact regions to be touched by the hand at the
same time. Based on the hand-object contact consistency, we design novel
objectives in training the human grasp generation model and also a new
self-supervised task which allows the grasp generation network to be adjusted
even during test time. Our experiments show significant improvement in human
grasp generation over state-of-the-art approaches by a large margin. More
interestingly, by optimizing the model during test time with the
self-supervised task, it helps achieve larger gain on unseen and out-of-domain
objects. Project page: this https URL

本文提出基于对象接触点和手接触点的一致性设计新目标模型以及自我监督任务来生成人类手的抓取姿态，实验证明该模型在人类手抓取方面比其他最先进方法有了显著的提升，特别是在测试过程中对未见过和超出领域的物体表现出更大的增益。