We propose to learn to generate grasping motion for manipulation with a
dexterous hand using implicit functions. With continuous time inputs, the model
can generate a continuous and smooth grasping plan. We name the proposed model
Continuous Grasping Function (CGF). CGF is learned via generative modeling with
a Conditional Variational Autoencoder using 3D human demonstrations. We will
first convert the large-scale human-object interaction trajectories to robot
demonstrations via motion retargeting, and then use these demonstrations to
train CGF. During inference, we perform sampling with CGF to generate different
grasping plans in the simulator and select the successful ones to transfer to
the real robot. By training on diverse human data, our CGF allows
generalization to manipulate multiple objects. Compared to previous planning
algorithms, CGF is more efficient and achieves significant improvement on
success rate when transferred to grasping with the real Allegro Hand. Our
project page is available at this https URL .

使用隐式函数学习生成灵巧手运动的抓取计划，结合条件变分自动编码器和 3D 人体演示进行生成建模，该模型名为连续抓取函数（CGF）。通过在模拟器进行不断采样，并选择成功的抓取计划转移到真实机器人手上，与之前的规划算法相比，CGF 更高效，成功率也显著提高。

从人类示范中学习灵巧手的连续抓取功能

Learning Continuous Grasping Function with a Dexterous Hand from Human  Demonstrations

We explore learning-based approaches for feedback control of a dexterous
five-finger hand performing non-prehensile manipulation. First, we learn local
controllers that are able to perform the task starting at a predefined initial
state. These controllers are constructed using trajectory optimization with
respect to locally-linear time-varying models learned directly from sensor
data. In some cases, we initialize the optimizer with human demonstrations
collected via teleoperation in a virtual environment. We demonstrate that such
controllers can perform the task robustly, both in simulation and on the
physical platform, for a limited range of initial conditions around the trained
starting state. We then consider two interpolation methods for generalizing to
a wider range of initial conditions: deep learning, and nearest neighbors. We
find that nearest neighbors achieve higher performance. Nevertheless, the
neural network has its advantages: it uses only tactile and proprioceptive
feedback but no visual feedback about the object (i.e. it performs the task
blind) and learns a time-invariant policy. In contrast, the nearest neighbors
method switches between time-varying local controllers based on the proximity
of initial object states sensed via motion capture. While both generalization
methods leave room for improvement, our work shows that (i) local
trajectory-based controllers for complex non-prehensile manipulation tasks can
be constructed from surprisingly small amounts of training data, and (ii)
collections of such controllers can be interpolated to form more global
controllers. Results are summarized in the supplementary video:
this https URL

本研究探索了学习控制方法对于机械手进行非抓握性操作的影响，通过深度学习和最近邻等方法实现了控制器的泛化。研究表明仅基于时间轨迹的控制器仅需要少量训练数据即可构建，同时多个控制器可以进行插值形成更全局的控制器。