Single-point annotation in visual tasks, with the goal of minimizing
labelling costs, is becoming increasingly prominent in research. Recently,
visual foundation models, such as Segment Anything (SAM), have gained
widespread usage due to their robust zero-shot capabilities and exceptional
annotation performance. However, SAM's class-agnostic output and high
confidence in local segmentation introduce 'semantic ambiguity', posing a
challenge for precise category-specific segmentation. In this paper, we
introduce a cost-effective category-specific segmenter using SAM. To tackle
this challenge, we have devised a Semantic-Aware Instance Segmentation Network
(SAPNet) that integrates Multiple Instance Learning (MIL) with matching
capability and SAM with point prompts. SAPNet strategically selects the most
representative mask proposals generated by SAM to supervise segmentation, with
a specific focus on object category information. Moreover, we introduce the
Point Distance Guidance and Box Mining Strategy to mitigate inherent
challenges: 'group' and 'local' issues in weakly supervised segmentation. These
strategies serve to further enhance the overall segmentation performance. The
experimental results on Pascal VOC and COCO demonstrate the promising
performance of our proposed SAPNet, emphasizing its semantic matching
capabilities and its potential to advance point-prompted instance segmentation.
The code will be made publicly available.

使用 Segment Anything (SAM) 集成 Multiple Instance Learning (MIL) 的 Semantic-Aware Instance Segmentation Network (SAPNet) 有助于解决弱监督分割中 'group' 和 'local' 问题，提高语义匹配能力和基于点提示的实例分割性能。