Semantic 3D scene understanding is a problem of critical importance in robotics. While significant advances have been made in simultaneous localization and mapping algorithms, robots are still far from having the common sense knowledge about household objects and their locations of an average human. We introduce a novel method for leveraging common sense embedded within large language models for labelling rooms given the objects contained within. This algorithm has the added benefits of (i) requiring no task-specific pre-training (operating entirely in the zero-shot regime) and (ii) generalizing to arbitrary room and object labels, including previously-unseen ones -- both of which are highly desirable traits in robotic scene understanding algorithms. The proposed algorithm operates on 3D scene graphs produced by modern spatial perception systems, and we hope it will pave the way to more generalizable and scalable high-level 3D scene understanding for robotics.

提出了一种使用大型语言模型中通用的常识知识为机器人场景中的物品进行标记，该算法不需要任务特定的预训练，并且可以泛化到任意房间和物品标签——这是机器人场景理解算法中非常理想的特征。该算法基于由现代空间感知系统产生的3D场景图，并希望将为机器人提供更具有可推广性和可扩展性的高级3D场景理解铺平道路。

从大型语言模型中提取零样机通用常识，用于机器人三维场景理解