Occupancy prediction tasks focus on the inference of both geometry and
semantic labels for each voxel, which is an important perception mission.
However, it is still a semantic segmentation task without distinguishing
various instances. Further, although some existing works, such as
Open-Vocabulary Occupancy (OVO), have already solved the problem of open
vocabulary detection, visual grounding in occupancy has not been solved to the
best of our knowledge. To tackle the above two limitations, this paper proposes
Occupancy Grounding (OG), a novel method that equips vanilla occupancy instance
segmentation ability and could operate visual grounding in a voxel manner with
the help of grounded-SAM. Keys to our approach are (1) affinity field
prediction for instance clustering and (2) association strategy for aligning 2D
instance masks and 3D occupancy instances. Extensive experiments have been
conducted whose visualization results and analysis are shown below. Our code
will be publicly released soon.

本篇研究提出一种称为 Occupancy Grounding (OG) 的新模型，该模型通过实例分割和视觉定位实现体素的可视化推理。该方法采用的关键策略是亲和力领域预测和对其进行实例聚类，并以对齐 2D 实例掩模和 3D 所占据的实例的关联策略为特征。