Advancements in 3D instance segmentation have traditionally been tethered to
the availability of annotated datasets, limiting their application to a narrow
spectrum of object categories. Recent efforts have sought to harness
vision-language models like CLIP for open-set semantic reasoning, yet these
methods struggle to distinguish between objects of the same categories and rely
on specific prompts that are not universally applicable. In this paper, we
introduce SAI3D, a novel zero-shot 3D instance segmentation approach that
synergistically leverages geometric priors and semantic cues derived from
Segment Anything Model (SAM). Our method partitions a 3D scene into geometric
primitives, which are then progressively merged into 3D instance segmentations
that are consistent with the multi-view SAM masks. Moreover, we design a
hierarchical region-growing algorithm with a dynamic thresholding mechanism,
which largely improves the robustness of finegrained 3D scene parsing.
Empirical evaluations on Scan-Net and the more challenging ScanNet++ datasets
demonstrate the superiority of our approach. Notably, SAI3D outperforms
existing open-vocabulary baselines and even surpasses fully-supervised methods
in class-agnostic segmentation on ScanNet++.

SAI3D 是一种新颖的零样本三维实例分割方法，通过整合来自 Segment Anything Model (SAM) 的几何先验和语义线索，将三维场景划分为几何原语，然后逐步合并为与多视角 SAM 掩码一致的三维实例分割。通过具有动态阈值机制的分层区域增长算法，进一步提升细粒度三维场景解析的鲁棒性。在 Scan-Net 和更具挑战性的 ScanNet++ 数据集上进行的实证评估表明，我们的方法优于现有的开放词汇基准线，并在 ScanNet++ 上的类别不可知分割中甚至超过了全监督方法。