3D decomposition/segmentation still remains a challenge as large-scale 3D
annotated data is not readily available. Contemporary approaches typically
leverage 2D machine-generated segments, integrating them for 3D consistency.
While the majority of these methods are based on NeRFs, they face a potential
weakness that the instance/semantic embedding features derive from independent
MLPs, thus preventing the segmentation network from learning the geometric
details of the objects directly through radiance and density. In this paper, we
propose ClusteringSDF, a novel approach to achieve both segmentation and
reconstruction in 3D via the neural implicit surface representation,
specifically Signal Distance Function (SDF), where the segmentation rendering
is directly integrated with the volume rendering of neural implicit surfaces.
Although based on ObjectSDF++, ClusteringSDF no longer requires the
ground-truth segments for supervision while maintaining the capability of
reconstructing individual object surfaces, but purely with the noisy and
inconsistent labels from pre-trained models.As the core of ClusteringSDF, we
introduce a high-efficient clustering mechanism for lifting the 2D labels to 3D
and the experimental results on the challenging scenes from ScanNet and Replica
datasets show that ClusteringSDF can achieve competitive performance compared
against the state-of-the-art with significantly reduced training time.

通过神经隐式表面表示，即信号距离函数（SDF），提出了 ClusteringSDF 方法，实现了 3D 中的分割和重建。这项研究通过高效的聚类机制将 2D 标签提升到 3D，并在 ScanNet 和 Replica 数据集上的挑战性场景中取得了与现有技术相竞争的表现，而且训练时间大大缩短。