Semantic 3D scene understanding is a problem of critical importance in
robotics. While significant advances have been made in spatial perception,
robots are still far from having the common-sense knowledge about household
objects and locations of an average human. We thus investigate the use of large
language models to impart common sense for scene understanding. Specifically,
we introduce three paradigms for leveraging language for classifying rooms in
indoor environments based on their contained objects: (i) a zero-shot approach,
(ii) a feed-forward classifier approach, and (iii) a contrastive classifier
approach. These methods operate on 3D scene graphs produced by modern spatial
perception systems. We then analyze each approach, demonstrating notable
zero-shot generalization and transfer capabilities stemming from their use of
language. Finally, we show these approaches also apply to inferring building
labels from contained rooms and demonstrate our zero-shot approach on a real
environment. All code can be found at
this https URL.

探讨使用大量语言模型来实现场景理解的常识；介绍了三种利用语言对包含对象的室内环境进行分类的范例：（i）零样本方法，（ii）前馈分类器方法和（iii）对比分类器方法，在现代空间感知系统生成的 3D 场景图上进行操作，并通过分析每种途径，展示了显着的零样本泛化和转移能力；最后，展示了这些方法也适用于推断包含房间的建筑标签，并在真实环境中展示了零样本方法。