Current point cloud semantic segmentation has achieved great advances when
given sufficient labels. However, the dense annotation of LiDAR point clouds
remains prohibitively expensive and time-consuming, unable to keep up with the
continuously growing volume of data. In this paper, we propose annotating
images with scattered points, followed by utilizing SAM (a Foundation model) to
generate semantic segmentation labels for the images. Finally, by mapping the
segmentation labels of the images to the LiDAR space using the intrinsic and
extrinsic parameters of the camera and LiDAR, we obtain labels for point cloud
semantic segmentation, and release Scatter-KITTI and Scatter-nuScenes, which
are the first works to utilize image segmentation-based SAM for weakly
supervised point cloud semantic segmentation. Furthermore, to mitigate the
influence of erroneous pseudo labels obtained from sparse annotations on point
cloud features, we propose a multi-modal weakly supervised network for LiDAR
semantic segmentation, called MM-ScatterNet. This network combines features
from both point cloud and image modalities, enhancing the representation
learning of point clouds by introducing consistency constraints between
multi-modal features and point cloud features. On the SemanticKITTI dataset, we
achieve 66\% of fully supervised performance using only 0.02% of annotated
data, and on the NuScenes dataset, we achieve 95% of fully supervised
performance using only 0.1% labeled points.

本文提出了使用散点图对图像进行注释，然后利用 SAM（一种基本模型）为图像生成语义分割标签。通过使用相机和 LiDAR 的内在和外在参数将图像的分割标签映射到 LiDAR 空间，我们获得了点云语义分割的标签，并发布了 Scatter-KITTI 和 Scatter-nuScenes，这是第一个利用基于图像分割的 SAM 进行弱监督点云语义分割的作品。此外，为了减小稀疏注释获取的错误伪标签对点云特征的影响，我们提出了一种用于 LiDAR 语义分割的多模态弱监督网络，称为 MM-ScatterNet。该网络通过引入多模态特征和点云特征之间的一致性约束，增强了点云的表示学习。在 SemanticKITTI 数据集上，我们仅使用 0.02％的注释数据就实现了 66％的全监督性能，在 NuScenes 数据集上，仅使用 0.1％的标记点就实现了 95％的全监督性能。