Bimanual manipulation is a longstanding challenge in robotics due to the
large number of degrees of freedom and the strict spatial and temporal
synchronization required to generate meaningful behavior. Humans learn bimanual
manipulation skills by watching other humans and by refining their abilities
through play. In this work, we aim to enable robots to learn bimanual
manipulation behaviors from human video demonstrations and fine-tune them
through interaction. Inspired by seminal work in psychology and biomechanics,
we propose modeling the interaction between two hands as a serial kinematic
linkage -- as a screw motion, in particular, that we use to define a new action
space for bimanual manipulation: screw actions. We introduce ScrewMimic, a
framework that leverages this novel action representation to facilitate
learning from human demonstration and self-supervised policy fine-tuning. Our
experiments demonstrate that ScrewMimic is able to learn several complex
bimanual behaviors from a single human video demonstration, and that it
outperforms baselines that interpret demonstrations and fine-tune directly in
the original space of motion of both arms. For more information and video
results, this https URL

利用人类示范视频和自我监督策略微调，我们提出了一种名为 ScrewMimic 的框架，通过将两只手之间的相互作用建模为串行运动链形式进行双手操作，从而实现机器人学习双手操作行为。实验结果表明，ScrewMimic 能够从单个人类示范视频中学习多种复杂的双手操作行为，并且在解释示范和直接在两只手的原始运动空间进行微调的基准方法上表现出更好的性能。