While there have been significant strides in dexterous manipulation, most of
it is limited to benchmark tasks like in-hand reorientation which are of
limited utility in the real world. The main benefit of dexterous hands over
two-fingered ones is their ability to pickup tools and other objects (including
thin ones) and grasp them firmly to apply force. However, this task requires
both a complex understanding of functional affordances as well as precise
low-level control. While prior work obtains affordances from human data this
approach doesn't scale to low-level control. Similarly, simulation training
cannot give the robot an understanding of real-world semantics. In this paper,
we aim to combine the best of both worlds to accomplish functional grasping for
in-the-wild objects. We use a modular approach. First, affordances are obtained
by matching corresponding regions of different objects and then a low-level
policy trained in sim is run to grasp it. We propose a novel application of
eigengrasps to reduce the search space of RL using a small amount of human data
and find that it leads to more stable and physically realistic motion. We find
that eigengrasp action space beats baselines in simulation and outperforms
hardcoded grasping in real and matches or outperforms a trained human
teleoperator. Results visualizations and videos at this https URL

本文旨在结合两种方法，实现对自然环境中物体的功能性抓取。首先，通过匹配不同物体的相应区域来获取各物体的可负担性，然后在模拟中运行一个经过训练的低级策略来抓取物体。我们提出了一种新颖的使用特征抓取来减少强化学习搜索空间的应用，使用少量人类数据实现更稳定和物理上逼真的运动。实验结果可在该 URL 中获得可视化和视频。