Driven by large-data pre-training, Segment Anything Model (SAM) has been
demonstrated as a powerful and promptable framework, revolutionizing the
segmentation models. Despite the generality, customizing SAM for specific
visual concepts without man-powered prompting is under explored, e.g.,
automatically segmenting your pet dog in different images. In this paper, we
propose a training-free Personalization approach for SAM, termed as PerSAM.
Given only a single image with a reference mask, PerSAM first localizes the
target concept by a location prior, and segments it within other images or
videos via three techniques: target-guided attention, target-semantic
prompting, and cascaded post-refinement. In this way, we effectively adapt SAM
for private use without any training. To further alleviate the mask ambiguity,
we present an efficient one-shot fine-tuning variant, PerSAM-F. Freezing the
entire SAM, we introduce two learnable weights for multi-scale masks, only
training 2 parameters within 10 seconds for improved performance. To
demonstrate our efficacy, we construct a new segmentation dataset, PerSeg, for
personalized evaluation, and test our methods on video object segmentation with
competitive performance. Besides, our approach can also enhance DreamBooth to
personalize Stable Diffusion for text-to-image generation, which discards the
background disturbance for better target appearance learning. Code is released
at this https URL

本文提出了一种称为 PerSAM 的、无需训练的个性化方法，该方法首先通过位置先验定位目标概念，然后通过三种技术 - 目标引导注意力、目标语义提示和级联后处理在其他图像或视频中对其进行分割，有效地适应 SAM 的私人使用。此外，我们还提出了一种高效的单次微调变体，PerSAM-F，以缓解掩模的歧义。我们构建了一个新的分割数据集 PerSeg，并在具有竞争性的性能的视频对象分割上测试了我们的方法。