Semantic segmentation is a key computer vision task that has been actively
researched for decades. In recent years, supervised methods have reached
unprecedented accuracy, however they require many pixel-level annotations for
every new class category which is very time-consuming and expensive.
Additionally, the ability of current semantic segmentation networks to handle a
large number of categories is limited. That means that images containing rare
class categories are unlikely to be well segmented by current methods. In this
paper we propose a novel approach for creating semantic segmentation masks for
every object, without the need for training segmentation networks or seeing any
segmentation masks. Our method takes as input the image-level labels of the
class categories present in the image; they can be obtained automatically or
manually. We utilize a vision-language embedding model (specifically CLIP) to
create a rough segmentation map for each class, using model interpretability
methods. We refine the maps using a test-time augmentation technique. The
output of this stage provides pixel-level pseudo-labels, instead of the manual
pixel-level labels required by supervised methods. Given the pseudo-labels, we
utilize single-image segmentation techniques to obtain high-quality output
segmentation masks. Our method is shown quantitatively and qualitatively to
outperform methods that use a similar amount of supervision. Our results are
particularly remarkable for images containing rare categories.

该论文提出一种基于视觉 - 语言嵌入模型和测试时增强技术的无监督伪标记方法，可以在不需要训练语义分割网络或看到任何分割掩码的情况下为每个对象创建语义分割遮罩。