We propose $\texttt{SAL}$ ($\texttt{S}$egment $\texttt{A}$nything in $\texttt{L}$idar) method consisting of a text-promptable zero-shot model for segmenting and classifying any object in Lidar, and a pseudo-labeling engine that facilitates model training without manual supervision. While the established paradigm for $\textit{Lidar Panoptic Segmentation}$ (LPS) relies on manual supervision for a handful of object classes defined a priori, we utilize 2D vision foundation models to generate 3D supervision "for free". Our pseudo-labels consist of instance masks and corresponding CLIP tokens, which we lift to Lidar using calibrated multi-modal data. By training our model on these labels, we distill the 2D foundation models into our Lidar $\texttt{SAL}$ model. Even without manual labels, our model achieves $91\%$ in terms of class-agnostic segmentation and $44\%$ in terms of zero-shot LPS of the fully supervised state-of-the-art. Furthermore, we outperform several baselines that do not distill but only lift image features to 3D. More importantly, we demonstrate that $\texttt{SAL}$ supports arbitrary class prompts, can be easily extended to new datasets, and shows significant potential to improve with increasing amounts of self-labeled data.

我们提出了一个名为SAL（Segment Anything in Lidar）的方法，包括一个可文本提示的零样本模型用于在Lidar中分割和分类任何物体，以及一个无需手动监督的伪标签引擎用于模型训练。通过利用2D视觉基础模型生成免费的3D监督，我们的模型在无需手动标签的情况下在类别无关分割上达到91％和零样本LPS上达到44％，超越了不使用蒸馏而仅将图像特征提升到3D的基线模型，并且支持任意类别提示，并且可以轻松地扩展到新数据集，并显示出随着自标记数据的增加而提高的潜力。

更好地呼唤 SAL：迈向 Lidar 中的任意物体分割学习