3D semantic occupancy prediction aims to obtain 3D fine-grained geometry and
semantics of the surrounding scene and is an important task for the robustness
of vision-centric autonomous driving. Most existing methods employ dense grids
such as voxels as scene representations, which ignore the sparsity of occupancy
and the diversity of object scales and thus lead to unbalanced allocation of
resources. To address this, we propose an object-centric representation to
describe 3D scenes with sparse 3D semantic Gaussians where each Gaussian
represents a flexible region of interest and its semantic features. We
aggregate information from images through the attention mechanism and
iteratively refine the properties of 3D Gaussians including position,
covariance, and semantics. We then propose an efficient Gaussian-to-voxel
splatting method to generate 3D occupancy predictions, which only aggregates
the neighboring Gaussians for a certain position. We conduct extensive
experiments on the widely adopted nuScenes and KITTI-360 datasets. Experimental
results demonstrate that GaussianFormer achieves comparable performance with
state-of-the-art methods with only 17.8% - 24.8% of their memory consumption.
Code is available at: this https URL

提出一种基于对象中心表示法的 3D 语义占位预测方法，通过稀疏 3D 语义高斯来描述场景，通过注意机制从图像中聚合信息并迭代细化 3D 高斯的属性，然后通过高斯到体素喷洒方法生成 3D 占位预测。