In autonomous vehicles, understanding the surrounding 3D environment of the
ego vehicle in real-time is essential. A compact way to represent scenes while
encoding geometric distances and semantic object information is via 3D semantic
occupancy maps. State of the art 3D mapping methods leverage transformers with
cross-attention mechanisms to elevate 2D vision-centric camera features into
the 3D domain. However, these methods encounter significant challenges in
real-time applications due to their high computational demands during
inference. This limitation is particularly problematic in autonomous vehicles,
where GPU resources must be shared with other tasks such as localization and
planning. In this paper, we introduce an approach that extracts features from
front-view 2D camera images and LiDAR scans, then employs a sparse convolution
network (Minkowski Engine), for 3D semantic occupancy prediction. Given that
outdoor scenes in autonomous driving scenarios are inherently sparse, the
utilization of sparse convolution is particularly apt. By jointly solving the
problems of 3D scene completion of sparse scenes and 3D semantic segmentation,
we provide a more efficient learning framework suitable for real-time
applications in autonomous vehicles. We also demonstrate competitive accuracy
on the nuScenes dataset.

自动驾驶车辆中，实时理解自身车辆周围的三维环境至关重要。本文介绍一种从前视二维相机图像和 LiDAR 扫描中提取特征并使用稀疏卷积网络（Minkowski Engine）进行三维语义占用预测的方法，以解决实时应用中的高计算需求和稀疏场景完整性的问题。