D semantic scene graphs are a powerful holistic representation as they
describe the individual objects and depict the relation between them. They are
compact high-level graphs that enable many tasks requiring scene reasoning. In
real-world settings, existing 3D estimation methods produce robust predictions
that mostly rely on dense inputs. In this work, we propose a real-time
framework that incrementally builds a consistent 3D semantic scene graph of a
scene given an RGB image sequence. Our method consists of a novel incremental
entity estimation pipeline and a scene graph prediction network. The proposed
pipeline simultaneously reconstructs a sparse point map and fuses entity
estimation from the input images. The proposed network estimates 3D semantic
scene graphs with iterative message passing using multi-view and geometric
features extracted from the scene entities. Extensive experiments on the 3RScan
dataset show the effectiveness of the proposed method in this challenging task,
outperforming state-of-the-art approaches.

本论文提出一种实时框架，使用 RGB 图像序列增量构建一致的三维语义场景图，包括新颖的增量实体估计流水线和场景图预测网络，使用多视角和几何特征，通过迭代信息传递估计三维语义场景图。经过在 3RScan 数据集上大量实验证明，该方法在这项挑战性任务中表现良好，优于现有的最先进方法。