Rearrangement tasks have been identified as a crucial challenge for
intelligent robotic manipulation, but few methods allow for precise
construction of unseen structures. We propose a visual foresight model for
pick-and-place rearrangement manipulation which is able to learn efficiently.
In addition, we develop a multi-modal action proposal module which builds on
the Goal-Conditioned Transporter Network, a state-of-the-art imitation learning
method. Our image-based task planning method, Transporters with Visual
Foresight, is able to learn from only a handful of data and generalize to
multiple unseen tasks in a zero-shot manner. TVF is able to improve the
performance of a state-of-the-art imitation learning method on unseen tasks in
simulation and real robot experiments. In particular, the average success rate
on unseen tasks improves from 55.4% to 78.5% in simulation experiments and from
30% to 63.3% in real robot experiments when given only tens of expert
demonstrations. Video and code are available on our project website:
this https URL

本文提出了一种基于图像的任务规划方法，称为具有视觉预见的运输器，旨在解决智能机器人操作中的重新排列任务。通过多模态动作建议模块和 Goal-Conditioned Transporter Network 的学习，实现零样本通用性和高效学习。在模拟和真实机器人实验中，该方法以很少的数据量显著提高了任务的成功率。