In this paper we tackle the problem of deformable object manipulation through
model-free visual reinforcement learning (RL). In order to circumvent the
sample inefficiency of RL, we propose two key ideas that accelerate learning.
First, we propose an iterative pick-place action space that encodes the
conditional relationship between picking and placing on deformable objects. The
explicit structural encoding enables faster learning under complex object
dynamics. Second, instead of jointly learning both the pick and the place
locations, we only explicitly learn the placing policy conditioned on random
pick points. Then, by selecting the pick point that has Maximal Value under
Placing (MVP), we obtain our picking policy. This provides us with an informed
picking policy during testing, while using only random pick points during
training. Experimentally, this learning framework obtains an order of magnitude
faster learning compared to independent action-spaces on our suite of
deformable object manipulation tasks with visual RGB observations. Finally,
using domain randomization, we transfer our policies to a real PR2 robot for
challenging cloth and rope coverage tasks, and demonstrate significant
improvements over standard RL techniques on average coverage.

本文介绍了基于无模型视觉强化学习的变形物体操作问题，通过提出迭代的拾取 - 放置空间和只显式学习放置策略来加速学习，并使用 MVP 选择策略，此学习框架在变形物体操纵任务方面获得了比独立空间快一个数量级的学习，并使用领域随机化将策略转移到 PR2 机器人上，对布料和绳索覆盖任务进行了实验验证。