Understanding the 3D semantics of a scene is a fundamental problem for various scenarios such as embodied agents. While NeRFs and 3DGS excel at novel-view synthesis, previous methods for understanding their semantics have been limited to incomplete 3D understanding: their segmentation results are 2D masks and their supervision is anchored at 2D pixels. This paper revisits the problem set to pursue a better 3D understanding of a scene modeled by NeRFs and 3DGS as follows. 1) We directly supervise the 3D points to train the language embedding field. It achieves state-of-the-art accuracy without relying on multi-scale language embeddings. 2) We transfer the pre-trained language field to 3DGS, achieving the first real-time rendering speed without sacrificing training time or accuracy. 3) We introduce a 3D querying and evaluation protocol for assessing the reconstructed geometry and semantics together. Code, checkpoints, and annotations will be available online. Project page: https://hyunji12.github.io/Open3DRF

本研究解决了之前方法在3D语义理解中的局限性，提出了一种通过直接监督3D点来训练语言嵌入场的创新方法，达到了最先进的准确度。此外，本研究首次实现了在保持训练时间和准确度的情况下，将预训练的语言场转移到3DGS上，从而实现实时渲染。最终引入的3D查询与评估协议为重建几何体和语义提供了新的评估标准。

重新思考3D空间中辐射场的开放词汇分割