We propose a system for rearranging objects in a scene to achieve a desired
object-scene placing relationship, such as a book inserted in an open slot of a
bookshelf. The pipeline generalizes to novel geometries, poses, and layouts of
both scenes and objects, and is trained from demonstrations to operate directly
on 3D point clouds. Our system overcomes challenges associated with the
existence of many geometrically-similar rearrangement solutions for a given
scene. By leveraging an iterative pose de-noising training procedure, we can
fit multi-modal demonstration data and produce multi-modal outputs while
remaining precise and accurate. We also show the advantages of conditioning on
relevant local geometric features while ignoring irrelevant global structure
that harms both generalization and precision. We demonstrate our approach on
three distinct rearrangement tasks that require handling multi-modality and
generalization over object shape and pose in both simulation and the real
world. Project website, code, and videos:
this https URL

我们提出了一个系统，用于重新排列场景中的物体以实现期望的物体 - 场景放置关系，例如将一本书插入书架的空槽中。该系统可以泛化到新的几何形状、姿态和场景、物体布局，并且通过从示范中直接在 3D 点云上操作来进行训练。我们的系统克服了对于给定场景存在许多几何上相似的重新排列解决方案的挑战。通过利用迭代式姿态去噪训练过程，我们可以适应多模态示范数据并产生多模态输出，同时保持精确和准确。我们还展示了通过关注相关的局部几何特征和忽略有害于泛化和精确性的不相关的全局结构的优点。我们在仿真和真实世界中展示了在处理多模态和对物体形状和姿态进行泛化时所需要的三个不同的重新排列任务的方法。项目网站、代码和视频：（此处为上面提供的链接）