Learning policies in simulation and transferring them to the real world has become a promising approach in dexterous manipulation. However, bridging the sim-to-real gap for each new task requires substantial human effort, such as careful reward engineering, hyperparameter tuning, and system identification. In this work, we present a system that leverages low-level skills to address these challenges for more complex tasks. Specifically, we introduce a hierarchical policy for in-hand object reorientation based on previously acquired rotation skills. This hierarchical policy learns to select which low-level skill to execute based on feedback from both the environment and the low-level skill policies themselves. Compared to learning from scratch, the hierarchical policy is more robust to out-of-distribution changes and transfers easily from simulation to real-world environments. Additionally, we propose a generalizable object pose estimator that uses proprioceptive information, low-level skill predictions, and control errors as inputs to estimate the object pose over time. We demonstrate that our system can reorient objects, including symmetrical and textureless ones, to a desired pose.

本研究解决了在模拟中学习策略并将其转移到现实世界所面临的挑战，特别是对于新任务，需要大量的人力努力。我们提出了一种基于之前获得的旋转技能的层次化策略，用于掌中物体重新定向，该策略能够根据环境反馈和低级技能政策选择执行的低级技能。研究表明，该系统在处理对象重新定向时具有更强的稳健性，并且能够有效地实现模拟到现实世界的转移。

从简单到复杂技能：掌中物体重新定向的案例