Multi-goal robot manipulation tasks with sparse rewards are difficult for
reinforcement learning (RL) algorithms due to the inefficiency in collecting
successful experiences. Recent algorithms such as Hindsight Experience Replay
(HER) expedite learning by taking advantage of failed trajectories and
replacing the desired goal with one of the achieved states so that any failed
trajectory can be utilized as a contribution to learning. However, HER
uniformly chooses failed trajectories, without taking into account which ones
might be the most valuable for learning. In this paper, we address this problem
and propose a novel approach Contact Energy Based Prioritization~(CEBP) to
select the samples from the replay buffer based on rich information due to
contact, leveraging the touch sensors in the gripper of the robot and object
displacement. Our prioritization scheme favors sampling of contact-rich
experiences, which are arguably the ones providing the largest amount of
information. We evaluate our proposed approach on various sparse reward robotic
tasks and compare them with the state-of-the-art methods. We show that our
method surpasses or performs on par with those methods on robot manipulation
tasks. Finally, we deploy the trained policy from our method to a real Franka
robot for a pick-and-place task. We observe that the robot can solve the task
successfully. The videos and code are publicly available at:
this https URL

本文介绍了一种基于接触能量的优先选择机制，用于解决强化学习中多目标机器人操作任务中稀疏奖励收集成功经验低效的问题，并在各种机器人操作任务中对该方法进行了评估。