Understanding the structural organisation of 3D indoor scenes in terms of
rooms is often accomplished via floorplan extraction. Robotic tasks such as
planning and navigation require a semantic understanding of the scene as well.
This is typically achieved via object-level semantic segmentation. However,
such methods struggle to segment out topological regions like "kitchen" in the
scene. In this work, we introduce a two-step pipeline. First, we extract a
topological map, i.e., floorplan of the indoor scene using a novel
multi-channel occupancy representation. Then, we generate CLIP-aligned features
and semantic labels for every room instance based on the objects it contains
using a self-attention transformer. Our language-topology alignment supports
natural language querying, e.g., a "place to cook" locates the "kitchen". We
outperform the current state-of-the-art on room segmentation by ~20% and room
classification by ~12%. Our detailed qualitative analysis and ablation studies
provide insights into the problem of joint structural and semantic 3D scene
understanding.

通过楼层平面图提取，我们引入了一个两步流程，首先使用一种新颖的多通道占用表示提取室内场景的拓扑图，然后使用自注意力转换器基于对象生成每个房间实例的 CLIP 对齐特征和语义标签。我们在房间分割和分类上的表现超过了当前最先进技术，我们的定性分析和消融研究揭示了联合结构和语义三维场景理解问题的见解。