Leveraging large-scale pre-training, vision foundational models showcase
notable performance benefits. While recent years have witnessed significant
advancements in segmentation algorithms, existing models still face challenges
to automatically segment personalized instances in dense and crowded scenarios.
The primary factor behind this limitation stems from bounding box-based
detections, which are constrained by occlusions, background clutter, and object
orientation, particularly when dealing with dense images. To this end, we
propose PerSense, an end-to-end, training-free, and model-agnostic one-shot
framework to address the personalized instance segmentation in dense images.
Towards developing this framework, we make following core contributions. (a) We
propose an Instance Detection Module (IDM) and leverage a Vision-Language
Model, a grounding object detector, and a few-shot object counter (FSOC) to
realize a new baseline. (b) To tackle false positives within candidate point
prompts, we design Point Prompt Selection Module (PPSM). Both IDM and PPSM
transform density maps from FSOC into personalized instance-level point prompts
for segmentation and offer a seamless integration in our model-agnostic
framework. (c) We introduce a feedback mechanism which enables PerSense to
harness the full potential of FSOC by automating the exemplar selection
process. (d) To promote algorithmic advances and effective tools for this
relatively underexplored task, we introduce PerSense-D, a dataset exclusive to
personalized instance segmentation in dense images. We validate the
effectiveness of PerSense on the task of personalized instance segmentation in
dense images on PerSense-D and comparison with SOTA. Additionally, our
qualitative findings demonstrate the adaptability of our framework to images
captured in-the-wild.

通过大规模预训练，基于视觉的基础模型展现了显著的性能优势；本文提出了 PerSense，这是一个端到端的、无需训练的、模型无关的一次性框架，用于解决稠密图像中的个性化实例分割问题，并通过验证和比较实验证明了其有效性和适应性。