Driven by large data trained segmentation models, such as SAM , research in
one-shot segmentation has experienced significant advancements. Recent
contributions like PerSAM and MATCHER , presented at ICLR 2024, utilize a
similar approach by leveraging SAM with one or a few reference images to
generate high quality segmentation masks for target images. Specifically, they
utilize raw encoded features to compute cosine similarity between patches
within reference and target images along the channel dimension, effectively
generating prompt points or boxes for the target images a technique referred to
as the matching strategy. However, relying solely on raw features might
introduce biases and lack robustness for such a complex task. To address this
concern, we delve into the issues of feature interaction and uneven
distribution inherent in raw feature based matching. In this paper, we propose
a simple and training-free method to enhance the validity and robustness of the
matching strategy at no additional computational cost (NubbleDrop). The core
concept involves randomly dropping feature channels (setting them to zero)
during the matching process, thereby preventing models from being influenced by
channels containing deceptive information. This technique mimics discarding
pathological nubbles, and it can be seamlessly applied to other similarity
computing scenarios. We conduct a comprehensive set of experiments, considering
a wide range of factors, to demonstrate the effectiveness and validity of our
proposed method. Our results showcase the significant improvements achieved
through this simmple and straightforward approach.

通过在一次分割中利用大数据训练的分割模型，如 SAM，我们提出了一种简单且无需额外计算成本的方法 (NubbleDrop) 来增强匹配策略的有效性和鲁棒性，通过随机丢弃特征通道来防止模型受到具有欺骗性信息的通道的影响。

NubbleDrop：一种简单的提升提示式单次分割匹配策略的方法

NubbleDrop: A Simple Way to Improve Matching Strategy for Prompted  One-Shot Segmentation

Powered by large-scale pre-training, vision foundation models exhibit
significant potential in open-world image understanding. Even though individual
models have limited capabilities, combining multiple such models properly can
lead to positive synergies and unleash their full potential. In this work, we
present Matcher, which segments anything with one shot by integrating an
all-purpose feature extraction model and a class-agnostic segmentation model.
Naively connecting the models results in unsatisfying performance, e.g., the
models tend to generate matching outliers and false-positive mask fragments. To
address these issues, we design a bidirectional matching strategy for accurate
cross-image semantic dense matching and a robust prompt sampler for mask
proposal generation. In addition, we propose a novel instance-level matching
strategy for controllable mask merging. The proposed Matcher method delivers
impressive generalization performance across various segmentation tasks, all
without training. For example, it achieves 52.7% mIoU on COCO-20$^i$ for
one-shot semantic segmentation, surpassing the state-of-the-art specialist
model by 1.6%. In addition, our visualization results show open-world
generality and flexibility on images in the wild. The code shall be released at
this https URL

Matcher 是一种结合通用特征提取和无类别分割模型实现一次性分割的方法，通过设计双向匹配策略和鲁棒的提示采样器进行交叉图像语义密集匹配以及实例级匹配策略，能够在各种分割任务中实现令人印象深刻的泛化性能，无需训练。