The goal of this work is to segment and name regions of images without access
to pixel-level labels during training. To tackle this task, we construct
segmenters by distilling the complementary strengths of two foundation models.
The first, CLIP (Radford et al. 2021), exhibits the ability to assign names to
image content but lacks an accessible representation of object structure. The
second, DINO (Caron et al. 2021), captures the spatial extent of objects but
has no knowledge of object names. Our method, termed NamedMask, begins by using
CLIP to construct category-specific archives of images. These images are
pseudo-labelled with a category-agnostic salient object detector bootstrapped
from DINO, then refined by category-specific segmenters using the CLIP archive
labels. Thanks to the high quality of the refined masks, we show that a
standard segmentation architecture trained on these archives with appropriate
data augmentation achieves impressive semantic segmentation abilities for both
single-object and multi-object images. As a result, our proposed NamedMask
performs favourably against a range of prior work on five benchmarks including
the VOC2012, COCO and large-scale ImageNet-S datasets.

本文提出 NamedMask 方法，采用 CLIP 和 DINO 两个模型相辅相成的特点，构建一种用于图像分割的模型，实现根据类别对图像进行语义分割，并在 VOC2012、COCO 和 ImageNet-S 等数据集上实验，取得了令人瞩目的结果。