Building 3D scene graphs has recently emerged as a topic in scene
representation for several embodied AI applications to represent the world in a
structured and rich manner. With their increased use in solving downstream
tasks (eg, navigation and room rearrangement), can we leverage and recycle them
for creating 3D maps of environments, a pivotal step in agent operation? We
focus on the fundamental problem of aligning pairs of 3D scene graphs whose
overlap can range from zero to partial and can contain arbitrary changes. We
propose SGAligner, the first method for aligning pairs of 3D scene graphs that
is robust to in-the-wild scenarios (ie, unknown overlap -- if any -- and
changes in the environment). We get inspired by multi-modality knowledge graphs
and use contrastive learning to learn a joint, multi-modal embedding space. We
evaluate on the 3RScan dataset and further showcase that our method can be used
for estimating the transformation between pairs of 3D scenes. Since benchmarks
for these tasks are missing, we create them on this dataset. The code,
benchmark, and trained models are available on the project website.

本文提出了 SGAligner 方法，基于多模态知识图谱和对比学习，实现了对两个 3D 场景图进行对齐的功能并且证明了其可用于估算 3D 场景之间的变换。

使用场景图进行三维场景对齐的 SGAligner

SGAligner : 3D Scene Alignment with Scene Graphs

In this work, we introduce the task of 3D object instance re-localization
(RIO): given one or multiple objects in an RGB-D scan, we want to estimate
their corresponding 6DoF poses in another 3D scan of the same environment taken
at a later point in time. We consider RIO a particularly important task in 3D
vision since it enables a wide range of practical applications, including
AI-assistants or robots that are asked to find a specific object in a 3D scene.
To address this problem, we first introduce 3RScan, a novel dataset and
benchmark, which features 1482 RGB-D scans of 478 environments across multiple
time steps. Each scene includes several objects whose positions change over
time, together with ground truth annotations of object instances and their
respective 6DoF mappings among re-scans. Automatically finding 6DoF object
poses leads to a particular challenging feature matching task due to varying
partial observations and changes in the surrounding context. To this end, we
introduce a new data-driven approach that efficiently finds matching features
using a fully-convolutional 3D correspondence network operating on multiple
spatial scales. Combined with a 6DoF pose optimization, our method outperforms
state-of-the-art baselines on our newly-established benchmark, achieving an
accuracy of 30.58%.

本研究介绍了三维物体实例重新定位任务，提出了使用新的数据驱动方法 —— 完全卷积三维对应网络和位姿优化来完成该任务，并使用新建立的 3RScan 数据集和基准测试来评估其性能，实现了 30.58% 的准确率。