Dexterous manipulation tasks involving contact-rich interactions pose a
significant challenge for both model-based control systems and imitation
learning algorithms. The complexity arises from the need for multi-fingered
robotic hands to dynamically establish and break contacts, balance
non-prehensile forces, and control large degrees of freedom. Reinforcement
learning (RL) offers a promising approach due to its general applicability and
capacity to autonomously acquire optimal manipulation strategies. However, its
real-world application is often hindered by the necessity to generate a large
number of samples, reset the environment, and obtain reward signals. In this
work, we introduce an efficient system for learning dexterous manipulation
skills with RL to alleviate these challenges. The main idea of our approach is
the integration of recent advances in sample-efficient RL and replay buffer
bootstrapping. This combination allows us to utilize data from different tasks
or objects as a starting point for training new tasks, significantly improving
learning efficiency. Additionally, our system completes the real-world training
cycle by incorporating learned resets via an imitation-based pickup policy as
well as learned reward functions, eliminating the need for manual resets and
reward engineering. We demonstrate the benefits of reusing past data as replay
buffer initialization for new tasks, for instance, the fast acquisition of
intricate manipulation skills in the real world on a four-fingered robotic
hand. (Videos: this https URL)

我们介绍了一种用于学习灵巧操作技能的高效系统，通过整合最近在样本有效强化学习和重放缓冲引导方面的进展，利用来自不同任务或物体的数据作为训练新任务的起点，显著提高学习效率，同时通过基于模仿的拾取策略和学习奖励函数，消除了手动重置和奖励工程的需求，并在四指机械手上的真实环境中展示了重用先前数据作为重放缓冲初始化的好处，例如在真实世界中快速获得复杂操作技能。