The recent Segment Anything Model (SAM) represents a big leap in scaling up
segmentation models, allowing for powerful zero-shot capabilities and flexible
prompting. Despite being trained with 1.1 billion masks, SAM's mask prediction
quality falls short in many cases, particularly when dealing with objects that
have intricate structures. We propose HQ-SAM, equipping SAM with the ability to
accurately segment any object, while maintaining SAM's original promptable
design, efficiency, and zero-shot generalizability. Our careful design reuses
and preserves the pre-trained model weights of SAM, while only introducing
minimal additional parameters and computation. We design a learnable
High-Quality Output Token, which is injected into SAM's mask decoder and is
responsible for predicting the high-quality mask. Instead of only applying it
on mask-decoder features, we first fuse them with early and final ViT features
for improved mask details. To train our introduced learnable parameters, we
compose a dataset of 44K fine-grained masks from several sources. HQ-SAM is
only trained on the introduced detaset of 44k masks, which takes only 4 hours
on 8 GPUs. We show the efficacy of HQ-SAM in a suite of 9 diverse segmentation
datasets across different downstream tasks, where 7 out of them are evaluated
in a zero-shot transfer protocol. Our code and models will be released at
this https URL

提出了一种名为 HQ-SAM 的模型，该模型在保持 Segment Anything Model（SAM）原始 zero-shot 设计，高效性和推广性的同时，赋予 SAM 精确切分任何对象的能力，通过深度融合输入的不同特征并引入可学习的高质量输出 Token，有效提高了遮罩细节。在多种下游任务的 9 个不同分割数据集中展示 HQ-SAM 的有效性，其中有 7 个采用了零 - shot 转移协议进行评估。