We propose a system for rearranging objects in a scene to achieve a desired
object-scene placing relationship, such as a book inserted in an open slot of a
bookshelf. The pipeline generalizes to novel geometries, poses, and layouts of
both scenes and objects, and is trained from demonstrations to operate directly
on 3D point clouds. Our system overcomes challenges associated with the
existence of many geometrically-similar rearrangement solutions for a given
scene. By leveraging an iterative pose de-noising training procedure, we can
fit multi-modal demonstration data and produce multi-modal outputs while
remaining precise and accurate. We also show the advantages of conditioning on
relevant local geometric features while ignoring irrelevant global structure
that harms both generalization and precision. We demonstrate our approach on
three distinct rearrangement tasks that require handling multi-modality and
generalization over object shape and pose in both simulation and the real
world. Project website, code, and videos:
this https URL

我们提出了一个系统，用于重新排列场景中的物体以实现期望的物体 - 场景放置关系，例如将一本书插入书架的空槽中。该系统可以泛化到新的几何形状、姿态和场景、物体布局，并且通过从示范中直接在 3D 点云上操作来进行训练。我们的系统克服了对于给定场景存在许多几何上相似的重新排列解决方案的挑战。通过利用迭代式姿态去噪训练过程，我们可以适应多模态示范数据并产生多模态输出，同时保持精确和准确。我们还展示了通过关注相关的局部几何特征和忽略有害于泛化和精确性的不相关的全局结构的优点。我们在仿真和真实世界中展示了在处理多模态和对物体形状和姿态进行泛化时所需要的三个不同的重新排列任务的方法。项目网站、代码和视频：（此处为上面提供的链接）

搁置、堆叠、悬挂：多模态重新布局的关系姿态扩散

Shelving, Stacking, Hanging: Relational Pose Diffusion for Multi-modal  Rearrangement

There has been a significant recent progress in the field of Embodied AI with
researchers developing models and algorithms enabling embodied agents to
navigate and interact within completely unseen environments. In this paper, we
propose a new dataset and baseline models for the task of Rearrangement. We
particularly focus on the task of Room Rearrangement: an agent begins by
exploring a room and recording objects' initial configurations. We then remove
the agent and change the poses and states (e.g., open/closed) of some objects
in the room. The agent must restore the initial configurations of all objects
in the room. Our dataset, named RoomR, includes 6,000 distinct rearrangement
settings involving 72 different object types in 120 scenes. Our experiments
show that solving this challenging interactive task that involves navigation
and object interaction is beyond the capabilities of the current
state-of-the-art techniques for embodied tasks and we are still very far from
achieving perfect performance on these types of tasks. The code and the dataset
are available at: this https URL

一项围绕着域中物理实体与环境互动的任务 - 室内重新配置的新数据集 RoomR 并结合实验展示现有的 Embodied AI 技术在此类任务方面性能仍有提升空间。

视觉房间重组

Visual Room Rearrangement

We describe a framework for research and evaluation in Embodied AI. Our
proposal is based on a canonical task: Rearrangement. A standard task can focus
the development of new techniques and serve as a source of trained models that
can be transferred to other settings. In the rearrangement task, the goal is to
bring a given physical environment into a specified state. The goal state can
be specified by object poses, by images, by a description in language, or by
letting the agent experience the environment in the goal state. We characterize
rearrangement scenarios along different axes and describe metrics for
benchmarking rearrangement performance. To facilitate research and exploration,
we present experimental testbeds of rearrangement scenarios in four different
simulation environments. We anticipate that other datasets will be released and
new simulation platforms will be built to support training of rearrangement
agents and their deployment on physical systems.

本文提出了一个以重新排列任务为基础的体现智能 (AI Embodied) 的研究和评价框架，并描述了四个不同的仿真环境中重新排列方案的实验测试床和性能度量标准，以支持重新排列代理的训练和部署。