Segment anything model (SAM) addresses two practical yet challenging
segmentation tasks: \textbf{segment anything (SegAny)}, which utilizes a
certain point to predict the mask for a single object of interest, and
\textbf{segment everything (SegEvery)}, which predicts the masks for all
objects on the image. What makes SegAny slow for SAM is its heavyweight image
encoder, which has been addressed by MobileSAM via decoupled knowledge
distillation. The efficiency bottleneck of SegEvery with SAM, however, lies in
its mask decoder because it needs to first generate numerous masks with
redundant grid-search prompts and then perform filtering to obtain the final
valid masks. We propose to improve its efficiency by directly generating the
final masks with only valid prompts, which can be obtained through object
discovery. Our proposed approach not only helps reduce the total time on the
mask decoder by at least 16 times but also achieves superior performance.
Specifically, our approach yields an average performance boost of 3.6\% (42.5\%
\textit{v.s.} 38.9\%) for zero-shot object proposal on the LVIS dataset with
the mask AR@$K$ metric. Qualitative results show that our approach generates
fine-grained masks while avoiding over-segmenting things. This project
targeting faster SegEvery than the original SAM is termed MobileSAMv2 to
differentiate from MobileSAM which targets faster SegAny. Moreover, we
demonstrate that our new prompt sampling is also compatible with the distilled
image encoders in MobileSAM, contributing to a unified framework for efficient
SegAny and SegEvery. The code is available at the same link as MobileSAM
Project
\href{https://github.com/ChaoningZhang/MobileSAM}{\textcolor{red}{this https URL}}.
\end{abstract}

通过对象发现直接生成有效提示来提高图像分割模型的效率和性能。