We consider the problem of sequential robotic manipulation of deformable
objects using tools. Previous works have shown that differentiable physics
simulators provide gradients to the environment state and help trajectory
optimization to converge orders of magnitude faster than model-free
reinforcement learning algorithms for deformable object manipulation. However,
such gradient-based trajectory optimization typically requires access to the
full simulator states and can only solve short-horizon, single-skill tasks due
to local optima. In this work, we propose a novel framework, named DiffSkill,
that uses a differentiable physics simulator for skill abstraction to solve
long-horizon deformable object manipulation tasks from sensory observations. In
particular, we first obtain short-horizon skills using individual tools from a
gradient-based optimizer, using the full state information in a differentiable
simulator; we then learn a neural skill abstractor from the demonstration
trajectories which takes RGBD images as input. Finally, we plan over the skills
by finding the intermediate goals and then solve long-horizon tasks. We show
the advantages of our method in a new set of sequential deformable object
manipulation tasks compared to previous reinforcement learning algorithms and
compared to the trajectory optimizer.

本研究提出了一个名为 DiffSkill 的新框架，使用可微分的物理模拟器进行技能抽象，从感官观测中解决长期目标可达的可变形物体操作任务。