Several unsupervised image segmentation approaches have been proposed which eliminate the need for dense manually-annotated segmentation masks; current models separately handle either semantic segmentation (e.g., STEGO) or class-agnostic instance segmentation (e.g., CutLER), but not both (i.e., panoptic segmentation). We propose an Unsupervised Universal Segmentation model (U2Seg) adept at performing various image segmentation tasks -- instance, semantic and panoptic -- using a novel unified framework. U2Seg generates pseudo semantic labels for these segmentation tasks via leveraging self-supervised models followed by clustering; each cluster represents different semantic and/or instance membership of pixels. We then self-train the model on these pseudo semantic labels, yielding substantial performance gains over specialized methods tailored to each task: a +2.6 AP$^{\text{box}}$ boost vs. CutLER in unsupervised instance segmentation on COCO and a +7.0 PixelAcc increase (vs. STEGO) in unsupervised semantic segmentation on COCOStuff. Moreover, our method sets up a new baseline for unsupervised panoptic segmentation, which has not been previously explored. U2Seg is also a strong pretrained model for few-shot segmentation, surpassing CutLER by +5.0 AP$^{\text{mask}}$ when trained on a low-data regime, e.g., only 1% COCO labels. We hope our simple yet effective method can inspire more research on unsupervised universal image segmentation.

我们提出了一种名为U2Seg的无监督通用分割模型，能够使用统一框架执行多种图像分割任务：实例分割、语义分割和全景分割。该方法通过利用自监督模型生成伪语义标签，并在此基础上进行自我训练，达到了与专门设计的方法相比显著的性能提升，同时也为无监督全景分割建立了一个新的基准。此外，在少样本情况下，即只有1%的COCO标签时，U2Seg也是强大的预训练模型，比CutLER在APmask上提升了5.0个点。我们希望我们简单而有效的方法能够激发更多关于无监督通用图像分割的研究。