Current 3D scene segmentation methods are heavily dependent on manually
annotated 3D training datasets. Such manual annotations are labor-intensive,
and often lack fine-grained details. Importantly, models trained on this data
typically struggle to recognize object classes beyond the annotated classes,
i.e., they do not generalize well to unseen domains and require additional
domain-specific annotations. In contrast, 2D foundation models demonstrate
strong generalization and impressive zero-shot abilities, inspiring us to
incorporate these characteristics from 2D models into 3D models. Therefore, we
explore the use of image segmentation foundation models to automatically
generate training labels for 3D segmentation. We propose Segment3D, a method
for class-agnostic 3D scene segmentation that produces high-quality 3D
segmentation masks. It improves over existing 3D segmentation models
(especially on fine-grained masks), and enables easily adding new training data
to further boost the segmentation performance -- all without the need for
manual training labels.

通过引入 2D 模型的特性，我们提出了 Segment3D 方法，用于生成高质量的 3D 场景分割掩码，从而实现对于细粒度掩码的改进，并且能够轻松添加新的训练数据以进一步提升分割性能，而无需手动标注训练标签。

Segment3D: 无需手动标注的细粒度类别无关三维分割学习

Segment3D: Learning Fine-Grained Class-Agnostic 3D Segmentation without  Manual Labels

Semantic annotations are indispensable to train or evaluate perception
models, yet very costly to acquire. This work introduces a fully automated
2D/3D labeling framework that, without any human intervention, can generate
labels for RGB-D scans at equal (or better) level of accuracy than comparable
manually annotated datasets such as ScanNet. Our approach is based on an
ensemble of state-of-the-art segmentation models and 3D lifting through neural
rendering. We demonstrate the effectiveness of our LabelMaker pipeline by
generating significantly better labels for the ScanNet datasets and
automatically labelling the previously unlabeled ARKitScenes dataset. Code and
models are available at this https URL

本文介绍了一种完全自动化的 2D/3D 标注框架，可以在无任何人为干预的情况下，为 RGB-D 扫描生成与可比较的手动注释的数据集（如 ScanNet）相同（或更高）准确度的标签，利用最先进的分割模型的集成和通过神经渲染进行的三维提升，我们的 LabelMaker 流水线的有效性通过为 ScanNet 数据集生成更好的标签并自动标注先前未标记的 ARKitScenes 数据集来进行了演示。

LABELMAKER：从 RGB-D 轨迹自动生成语义标签

LABELMAKER: Automatic Semantic Label Generation from RGB-D Trajectories

In computational pathology, automatic nuclei instance segmentation plays an
essential role in whole slide image analysis. While many computerized
approaches have been proposed for this task, supervised deep learning (DL)
methods have shown superior segmentation performances compared to classical
machine learning and image processing techniques. However, these models need
fully annotated datasets for training which is challenging to acquire,
especially in the medical domain. In this work, we release one of the biggest
fully manually annotated datasets of nuclei in Hematoxylin and Eosin
(H&E)-stained histological images, called NuInsSeg. This dataset contains 665
image patches with more than 30,000 manually segmented nuclei from 31 human and
mouse organs. Moreover, for the first time, we provide additional ambiguous
area masks for the entire dataset. These vague areas represent the parts of the
images where precise and deterministic manual annotations are impossible, even
for human experts. The dataset and detailed step-by-step instructions to
generate related segmentation masks are publicly available at
this https URL and
this https URL, respectively.

我们发布了一个大规模的手工标注的细胞核数据集 NuInsSeg，其中包含来自 31 个人和小鼠器官的 665 个图像补丁，超过 30,000 个手动分割的细胞核。此外，我们首次提供了整个数据集的模糊区域掩模，用于表示图像的某些部分即使对于人类专家来说也无法精确和确定地进行手动注释。