Panoptic occupancy poses a novel challenge by aiming to integrate instance
occupancy and semantic occupancy within a unified framework. However, there is
still a lack of efficient solutions for panoptic occupancy. In this paper, we
propose Panoptic-FlashOcc, a straightforward yet robust 2D feature framework
that enables realtime panoptic occupancy. Building upon the lightweight design
of FlashOcc, our approach simultaneously learns semantic occupancy and
class-aware instance clustering in a single network, these outputs are jointly
incorporated through panoptic occupancy procession for panoptic occupancy. This
approach effectively addresses the drawbacks of high memory and computation
requirements associated with three-dimensional voxel-level representations.
With its straightforward and efficient design that facilitates easy deployment,
Panoptic-FlashOcc demonstrates remarkable achievements in panoptic occupancy
prediction. On the Occ3D-nuScenes benchmark, it achieves exceptional
performance, with 38.5 RayIoU and 29.1 mIoU for semantic occupancy, operating
at a rapid speed of 43.9 FPS. Furthermore, it attains a notable score of 16.0
RayPQ for panoptic occupancy, accompanied by a fast inference speed of 30.2
FPS. These results surpass the performance of existing methodologies in terms
of both speed and accuracy. The source code and trained models can be found at
the following github repository: this https URL

Panoptic-FlashOcc 是一个简单而强大的 2D 特征框架，通过在单个网络中学习语义占用和类感知实例聚类，以实时实现全景占用，提高速度和准确性。

Panoptic-FlashOcc: 通过实例中心将语义空间与全景融合的高效基准

Panoptic-FlashOcc: An Efficient Baseline to Marry Semantic Occupancy  with Panoptic via Instance Center

Semantic occupancy has recently gained significant traction as a prominent 3D
scene representation. However, most existing methods rely on large and costly
datasets with fine-grained 3D voxel labels for training, which limits their
practicality and scalability, increasing the need for self-monitored learning
in this domain. In this work, we present a novel approach to occupancy
estimation inspired by neural radiance field (NeRF) using only 2D labels, which
are considerably easier to acquire. In particular, we employ differentiable
volumetric rendering to predict depth and semantic maps and train a 3D network
based on 2D supervision only. To enhance geometric accuracy and increase the
supervisory signal, we introduce temporal rendering of adjacent time steps.
Additionally, we introduce occupancy flow as a mechanism to handle dynamic
objects in the scene and ensure their temporal consistency. Through extensive
experimentation we demonstrate that 2D supervision only is sufficient to
achieve state-of-the-art performance compared to methods using 3D labels, while
outperforming concurrent 2D approaches. When combining 2D supervision with 3D
labels, temporal rendering and occupancy flow we outperform all previous
occupancy estimation models significantly. We conclude that the proposed
rendering supervision and occupancy flow advances occupancy estimation and
further bridges the gap towards self-supervised learning in this domain.

通过利用 2D 标签，我们提出了一种基于神经辐射场（NeRF）的新方法来估计体素占据，结合时间渲染和占据流，进一步推进了体素占据预测，并且在该领域中取得了最先进的性能。