Casting semantic segmentation of outdoor LiDAR point clouds as a 2D problem,
e.g., via range projection, is an effective and popular approach. These
projection-based methods usually benefit from fast computations and, when
combined with techniques which use other point cloud representations, achieve
state-of-the-art results. Today, projection-based methods leverage 2D CNNs but
recent advances in computer vision show that vision transformers (ViTs) have
achieved state-of-the-art results in many image-based benchmarks. In this work,
we question if projection-based methods for 3D semantic segmentation can
benefit from these latest improvements on ViTs. We answer positively but only
after combining them with three key ingredients: (a) ViTs are notoriously hard
to train and require a lot of training data to learn powerful representations.
By preserving the same backbone architecture as for RGB images, we can exploit
the knowledge from long training on large image collections that are much
cheaper to acquire and annotate than point clouds. We reach our best results
with pre-trained ViTs on large image datasets. (b) We compensate ViTs' lack of
inductive bias by substituting a tailored convolutional stem for the classical
linear embedding layer. (c) We refine pixel-wise predictions with a
convolutional decoder and a skip connection from the convolutional stem to
combine low-level but fine-grained features of the the convolutional stem with
the high-level but coarse predictions of the ViT encoder. With these
ingredients, we show that our method, called RangeViT, outperforms existing
projection-based methods on nuScenes and SemanticKITTI. We provide the
implementation code at this https URL.

本文应用 Vision Transformers 和卷积解码器改进了基于投影的 3D 语义分割方法以获得更好的结果，并在 nuScenes 和 SemanticKITTI 数据集上超过了现有的投影方法。