Using visual model-based learning for deformable object manipulation is
challenging due to difficulties in learning plannable visual representations
along with complex dynamic models. In this work, we propose a new learning
framework that jointly optimizes both the visual representation model and the
dynamics model using contrastive estimation. Using simulation data collected by
randomly perturbing deformable objects on a table, we learn latent dynamics
models for these objects in an offline fashion. Then, using the learned models,
we use simple model-based planning to solve challenging deformable object
manipulation tasks such as spreading ropes and cloths. Experimentally, we show
substantial improvements in performance over standard model-based learning
techniques across our rope and cloth manipulation suite. Finally, we transfer
our visual manipulation policies trained on data purely collected in simulation
to a real PR2 robot through domain randomization.

本研究提出了一种新的学习框架，使用对比估计来同时优化可视化表达模型和动态模型，在随机扰动变形对象并收集了模拟数据后，离线学习这些对象的潜在动态模型，随后使用简单的模型规划来解决具有挑战性的变形对象操纵任务，实验结果表明其性能优于传统模型学习技术，并通过域随机化将其可视化操纵策略从模拟数据成功地迁移到现实中的 PR2 机器人中。