The Segmentation Anything Model (SAM) requires labor-intensive data labeling.
We present Unsupervised SAM (UnSAM) for promptable and automatic whole-image
segmentation that does not require human annotations. UnSAM utilizes a
divide-and-conquer strategy to "discover" the hierarchical structure of visual
scenes. We first leverage top-down clustering methods to partition an unlabeled
image into instance/semantic level segments. For all pixels within a segment, a
bottom-up clustering method is employed to iteratively merge them into larger
groups, thereby forming a hierarchical structure. These unsupervised
multi-granular masks are then utilized to supervise model training. Evaluated
across seven popular datasets, UnSAM achieves competitive results with the
supervised counterpart SAM, and surpasses the previous state-of-the-art in
unsupervised segmentation by 11% in terms of AR. Moreover, we show that
supervised SAM can also benefit from our self-supervised labels. By integrating
our unsupervised pseudo masks into SA-1B's ground-truth masks and training
UnSAM with only 1% of SA-1B, a lightly semi-supervised UnSAM can often segment
entities overlooked by supervised SAM, exceeding SAM's AR by over 6.7% and AP
by 3.9% on SA-1B.

使用无监督 SAM 模型进行图像分割，通过将图像分割为实例 / 语义级别的片段并形成层次结构的无监督多粒度蒙版，提供了与有监督方法相媲美甚至更好的分割结果，并为有监督模型提供了改进的自我监督标签。