3D occupancy perception holds a pivotal role in recent vision-centric
autonomous driving systems by converting surround-view images into integrated
geometric and semantic representations within dense 3D grids. Nevertheless,
current models still encounter two main challenges: modeling depth accurately
in the 2D-3D view transformation stage, and overcoming the lack of
generalizability issues due to sparse LiDAR supervision. To address these
issues, this paper presents GEOcc, a Geometric-Enhanced Occupancy network
tailored for vision-only surround-view perception. Our approach is three-fold:
1) Integration of explicit lift-based depth prediction and implicit
projection-based transformers for depth modeling, enhancing the density and
robustness of view transformation. 2) Utilization of mask-based encoder-decoder
architecture for fine-grained semantic predictions; 3) Adoption of
context-aware self-training loss functions in the pertaining stage to
complement LiDAR supervision, involving the re-rendering of 2D depth maps from
3D occupancy features and leveraging image reconstruction loss to obtain denser
depth supervision besides sparse LiDAR ground-truths. Our approach achieves
State-Of-The-Art performance on the Occ3D-nuScenes dataset with the least image
resolution needed and the most weightless image backbone compared with current
models, marking an improvement of 3.3% due to our proposed contributions.
Comprehensive experimentation also demonstrates the consistent superiority of
our method over baselines and alternative approaches.

本文提出了 GEOcc，一种适用于仅使用视觉的全景视图感知的几何增强占用网络，它通过将环视图图像转换为密集的三维网格内的集成几何和语义表示来实现对三维占用的感知，解决了目前模型面临的两个主要挑战：在二维至三维视图变换阶段准确建模深度，以及由于稀疏的 LiDAR 监督引起的泛化问题的缺乏。

GEOcc：几何增强的三维占据网络与隐式 - 显式深度融合和上下文自监督

GEOcc: Geometrically Enhanced 3D Occupancy Network with  Implicit-Explicit Depth Fusion and Contextual Self-Supervision

3D occupancy perception technology aims to observe and understand dense 3D
environments for autonomous vehicles. Owing to its comprehensive perception
capability, this technology is emerging as a trend in autonomous driving
perception systems, and is attracting significant attention from both industry
and academia. Similar to traditional bird's-eye view (BEV) perception, 3D
occupancy perception has the nature of multi-source input and the necessity for
information fusion. However, the difference is that it captures vertical
structures that are ignored by 2D BEV. In this survey, we review the most
recent works on 3D occupancy perception, and provide in-depth analyses of
methodologies with various input modalities. Specifically, we summarize general
network pipelines, highlight information fusion techniques, and discuss
effective network training. We evaluate and analyze the occupancy perception
performance of the state-of-the-art on the most popular datasets. Furthermore,
challenges and future research directions are discussed. We hope this report
will inspire the community and encourage more research work on 3D occupancy
perception. A comprehensive list of studies in this survey is available in an
active repository that continuously collects the latest work:
this https URL

3D 占用感知技术是观察和理解自动驾驶车辆的密集 3D 环境的一项目标。本调查报告回顾了最新的 3D 占用感知研究，提供了对各种输入模态的方法论的深入分析，总结了网络流程、信息融合技术以及有效的网络训练，评估了现有算法在最流行的数据集上的性能，并讨论了挑战和未来研究方向，希望激发学术界对 3D 占用感知的更多研究工作。