Advancements in 3D instance segmentation have traditionally been tethered to the availability of annotated datasets, limiting their application to a narrow spectrum of object categories. Recent efforts have sought to harness vision-language models like CLIP for open-set semantic reasoning, yet these methods struggle to distinguish between objects of the same categories and rely on specific prompts that are not universally applicable. In this paper, we introduce SAI3D, a novel zero-shot 3D instance segmentation approach that synergistically leverages geometric priors and semantic cues derived from Segment Anything Model (SAM). Our method partitions a 3D scene into geometric primitives, which are then progressively merged into 3D instance segmentations that are consistent with the multi-view SAM masks. Moreover, we design a hierarchical region-growing algorithm with a dynamic thresholding mechanism, which largely improves the robustness of finegrained 3D scene parsing. Empirical evaluations on Scan-Net and the more challenging ScanNet++ datasets demonstrate the superiority of our approach. Notably, SAI3D outperforms existing open-vocabulary baselines and even surpasses fully-supervised methods in class-agnostic segmentation on ScanNet++.

SAI3D是一种新颖的零样本三维实例分割方法，通过整合来自Segment Anything Model (SAM)的几何先验和语义线索，将三维场景划分为几何原语，然后逐步合并为与多视角SAM掩码一致的三维实例分割。通过具有动态阈值机制的分层区域增长算法，进一步提升细粒度三维场景解析的鲁棒性。在Scan-Net和更具挑战性的ScanNet++数据集上进行的实证评估表明，我们的方法优于现有的开放词汇基准线，并在ScanNet++上的类别不可知分割中甚至超过了全监督方法。

SAI3D：在3D场景中分割任意实例