Point cloud data labeling is considered a time-consuming and expensive task
in autonomous driving, whereas unsupervised learning can avoid it by learning
point cloud representations from unannotated data. In this paper, we propose
UOV, a novel 3D Unsupervised framework assisted by 2D Open-Vocabulary
segmentation models. It consists of two stages: In the first stage, we
innovatively integrate high-quality textual and image features of 2D
open-vocabulary models and propose the Tri-Modal contrastive Pre-training
(TMP). In the second stage, spatial mapping between point clouds and images is
utilized to generate pseudo-labels, enabling cross-modal knowledge
distillation. Besides, we introduce the Approximate Flat Interaction (AFI) to
address the noise during alignment and label confusion. To validate the
superiority of UOV, extensive experiments are conducted on multiple related
datasets. We achieved a record-breaking 47.73% mIoU on the annotation-free
point cloud segmentation task in nuScenes, surpassing the previous best model
by 10.70% mIoU. Meanwhile, the performance of fine-tuning with 1% data on
nuScenes and SemanticKITTI reached a remarkable 51.75% mIoU and 48.14% mIoU,
outperforming all previous pre-trained models.

提出了一种由 2D 开放词汇的分割模型辅助的新型三维无监督框架（UOV），通过学习未标注数据的点云表征，采用了两个阶段：首先，创新地结合了 2D 开放词汇模型的高质量文本和图像特征，提出了三模态对比预训练（TMP）；其次，利用点云和图像之间的空间映射生成伪标签，实现异态知识蒸馏。并引入了近似平面交互（AFI）来解决对齐噪声和标签混淆问题。在多个相关数据集上进行了大量实验证明了 UOV 的优越性，在 nuScenes 的无需标注的点云分割任务上，实现了创纪录的 47.73% 的 mIoU，超过了之前最好模型 10.70% 的 mIoU。同时，在 nuScenes 和 SemanticKITTI 上使用 1% 的数据进行微调，取得了显著的 51.75% 和 48.14% 的 mIoU，超过了所有之前的预训练模型。