Mastering robotic manipulation skills through reinforcement learning (RL)
typically requires the design of shaped reward functions. Recent developments
in this area have demonstrated that using sparse rewards, i.e. rewarding the
agent only when the task has been successfully completed, can lead to better
policies. However, state-action space exploration is more difficult in this
case. Recent RL approaches to learning with sparse rewards have leveraged
high-quality human demonstrations for the task, but these can be costly, time
consuming or even impossible to obtain. In this paper, we propose a novel and
effective approach that does not require human demonstrations. We observe that
every robotic manipulation task could be seen as involving a locomotion task
from the perspective of the object being manipulated, i.e. the object could
learn how to reach a target state on its own. In order to exploit this idea, we
introduce a framework whereby an object locomotion policy is initially obtained
using a realistic physics simulator. This policy is then used to generate
auxiliary rewards, called simulated locomotion demonstration rewards (SLDRs),
which enable us to learn the robot manipulation policy. The proposed approach
has been evaluated on 13 tasks of increasing complexity, and can achieve higher
success rate and faster learning rates compared to alternative algorithms.
SLDRs are especially beneficial for tasks like multi-object stacking and
non-rigid object manipulation.

本文提出了一种新的机器人操作方法，该方法利用了物体本身的运动学习，通过使用物理模拟器中的对象运动策略生成辅助奖励，称为模拟运动演示奖励（SLDRs），该方法可以在不需要人类演示或昂贵成本的情况下，通过强化学习来掌握机器人操作技能，从而实现多物体堆放和非刚性物体操作等任务的更高成功率和更快学习。