The development of 2D foundation models for image segmentation has been
significantly advanced by the Segment Anything Model (SAM). However, achieving
similar success in 3D models remains a challenge due to issues such as
non-unified data formats, lightweight models, and the scarcity of labeled data
with diverse masks. To this end, we propose a 3D promptable segmentation model
(Point-SAM) focusing on point clouds. Our approach utilizes a transformer-based
method, extending SAM to the 3D domain. We leverage part-level and object-level
annotations and introduce a data engine to generate pseudo labels from SAM,
thereby distilling 2D knowledge into our 3D model. Our model outperforms
state-of-the-art models on several indoor and outdoor benchmarks and
demonstrates a variety of applications, such as 3D annotation. Codes and demo
can be found at this https URL

通过使用基于 Transformer 的方法在 3D 空间中扩展了 Segment Anything Model（SAM），我们提出了一种 3D 可提示分割模型（Point-SAM），利用部分级和对象级注释，从 SAM 中生成伪标签，将 2D 知识提取到我们的 3D 模型中，并在室内和室外基准测试中优于最先进的模型，展示了多种应用，如 3D 标注。