Modern methods for vision-centric autonomous driving perception widely adopt
the bird's-eye-view (BEV) representation to describe a 3D scene. Despite its
better efficiency than voxel representation, it has difficulty describing the
fine-grained 3D structure of a scene with a single plane. To address this, we
propose a tri-perspective view (TPV) representation which accompanies BEV with
two additional perpendicular planes. We model each point in the 3D space by
summing its projected features on the three planes. To lift image features to
the 3D TPV space, we further propose a transformer-based TPV encoder
(TPVFormer) to obtain the TPV features effectively. We employ the attention
mechanism to aggregate the image features corresponding to each query in each
TPV plane. Experiments show that our model trained with sparse supervision
effectively predicts the semantic occupancy for all voxels. We demonstrate for
the first time that using only camera inputs can achieve comparable performance
with LiDAR-based methods on the LiDAR segmentation task on nuScenes. Code:
this https URL.

采用鸟瞰图（BEV）描述自动驾驶中的三维场景难以描绘细粒度的三维结构，因此我们提出了三面图（TPV）表示法，并使用基于注意力机制的 TPV 编码器实现了显著的提升。模型可以通过稀疏监督有效预测语义占用，仅使用相机输入在 LiDAR 分割任务上可实现与基于 LiDAR 的方法相当的性能。