Recently, many 2D pretrained foundational models have demonstrated impressive
zero-shot prediction capabilities. In this work, we design a novel pipeline for
zero-shot 3D part segmentation, called ZeroPS. It high-quality transfers
knowledge from 2D pretrained foundational models to 3D point clouds. The main
idea of our approach is to explore the natural relationship between multi-view
correspondences and the prompt mechanism of foundational models and build
bridges on it. Our pipeline consists of two components: 1) a self-extension
component that extends 2D groups from a single viewpoint to spatial
global-level 3D groups; 2) a multi-modal labeling component that introduces a
two-dimensional checking mechanism to vote each 2D predicted bounding box to
the best matching 3D part, and a Class Non-highest Vote Penalty function to
refine the Vote Matrix. Additionally, a merging algorithm is included to merge
part-level 3D groups. Extensive evaluation of three zero-shot segmentation
tasks on PartnetE datasets, achieving state-of-the-art results with significant
improvements (+19.6%, +5.2% and +4.9%, respectively) over existing methods. Our
proposed approach does not need any training, fine-tuning or learnable
parameters. It is hardly affected by domain shift. The code will be released.

我们设计了一种新的零样本三维部分分割流水线 ZeroPS，它可以将二维预训练基础模型的知识高质量地传递到三维点云中。我们的方法主要基于多视图对应和基础模型的提示机制之间的自然关系，并在此基础上建立桥梁。我们的流水线包括两个组件：1) 自扩展组件，将单个视角的二维组扩展到三维全局级组；2) 多模态标记组件，引入二维检查机制，将每个二维预测的边界框投票至最佳匹配的三维部件，并使用类非最高投票惩罚函数来优化投票矩阵。此外，还包括一种合并算法来合并部件级的三维组。在 PartnetE 数据集上对三个零样本分割任务进行了广泛评估，相比现有方法，取得了显著改进的最新成果（分别提高了 19.6％，5.2％和 4.9％）。我们的方法不需要任何训练、微调或可学习参数，并且几乎不受领域变化的影响。代码将公开发布。