Dexterous manipulation with a multi-finger hand is one of the most
challenging problems in robotics. While recent progress in imitation learning
has largely improved the sample efficiency compared to Reinforcement Learning,
the learned policy can hardly generalize to manipulate novel objects, given
limited expert demonstrations. In this paper, we propose to learn dexterous
manipulation using large-scale demonstrations with diverse 3D objects in a
category, which are generated from a human grasp affordance model. This
generalizes the policy to novel object instances within the same category. To
train the policy, we propose a novel imitation learning objective jointly with
a geometric representation learning objective using our demonstrations. By
experimenting with relocating diverse objects in simulation, we show that our
approach outperforms baselines with a large margin when manipulating novel
objects. We also ablate the importance on 3D object representation learning for
manipulation. We include videos, code, and additional information on the
project website - this https URL .

本研究提出了使用大规模演示学习熟练操作多指手来处理机器人学中最具挑战性的问题。 主要采用人类抓握可行性模型生成 3D 物体演示，以实现策略的泛化。同时，本研究使用了新颖的模仿学习目标，以及几何表示学习目标，通过在仿真系统中重新定位多样的对象以证明本方法的有效性。