We propose $\texttt{SAL}$ ($\texttt{S}$egment $\texttt{A}$nything in
$\texttt{L}$idar) method consisting of a text-promptable zero-shot model for
segmenting and classifying any object in Lidar, and a pseudo-labeling engine
that facilitates model training without manual supervision. While the
established paradigm for $\textit{Lidar Panoptic Segmentation}$ (LPS) relies on
manual supervision for a handful of object classes defined a priori, we utilize
2D vision foundation models to generate 3D supervision "for free". Our
pseudo-labels consist of instance masks and corresponding CLIP tokens, which we
lift to Lidar using calibrated multi-modal data. By training our model on these
labels, we distill the 2D foundation models into our Lidar $\texttt{SAL}$
model. Even without manual labels, our model achieves $91\%$ in terms of
class-agnostic segmentation and $44\%$ in terms of zero-shot LPS of the fully
supervised state-of-the-art. Furthermore, we outperform several baselines that
do not distill but only lift image features to 3D. More importantly, we
demonstrate that $\texttt{SAL}$ supports arbitrary class prompts, can be easily
extended to new datasets, and shows significant potential to improve with
increasing amounts of self-labeled data.

我们提出了一个名为 SAL（Segment Anything in Lidar）的方法，包括一个可文本提示的零样本模型用于在 Lidar 中分割和分类任何物体，以及一个无需手动监督的伪标签引擎用于模型训练。通过利用 2D 视觉基础模型生成免费的 3D 监督，我们的模型在无需手动标签的情况下在类别无关分割上达到 91％和零样本 LPS 上达到 44％，超越了不使用蒸馏而仅将图像特征提升到 3D 的基线模型，并且支持任意类别提示，并且可以轻松地扩展到新数据集，并显示出随着自标记数据的增加而提高的潜力。