Occupancy prediction plays a pivotal role in the realm of autonomous driving.
Previous methods typically constructs a dense 3D volume, neglecting the
inherent sparsity of the scene, which results in a high computational cost.
Furthermore, these methods are limited to semantic occupancy and fail to
differentiate between distinct instances. To exploit the sparsity property and
ensure instance-awareness, we introduce a novel fully sparse panoptic occupancy
network, termed SparseOcc. SparseOcc initially reconstructs a sparse 3D
representation from visual inputs. Subsequently, it employs sparse instance
queries to predict each object instance from the sparse 3D representation.
These instance queries interact with 2D features via mask-guided sparse
sampling, thereby circumventing the need for costly dense features or global
attention. Additionally, we have established the first-ever vision-centric
panoptic occupancy benchmark. SparseOcc demonstrates its efficacy on the
Occ3D-nus dataset by achieving a mean Intersection over Union (mIoU) of 26.0,
while maintaining a real-time inference speed of 25.4 FPS. By incorporating
temporal modeling from the preceding 8 frames, SparseOcc further improves its
performance, achieving 30.9 mIoU without whistles and bells. Code will be made
available.

稀疏三维表示的实例感知的全稀疏全景占用网络 (SparseOcc) 通过稀疏的实例查询和基于掩模引导的稀疏采样，以及在先前 8 帧的时间建模的融合，实现了自主驾驶的占用预测，同时在 Occ3D-nus 数据集上达到了 26.0 的平均交并比 (mIoU)，并保持着 25.4 FPS 的实时推理速度。