This work presents a tuning-free semantic segmentation framework based on
classifying SAM masks by CLIP, which is universally applicable to various types
of supervision. Initially, we utilize CLIP's zero-shot classification ability
to generate pseudo-labels or perform open-vocabulary segmentation. However, the
misalignment between mask and CLIP text embeddings leads to suboptimal results.
To address this issue, we propose discrimination-bias aligned CLIP to closely
align mask and text embedding, offering an overhead-free performance gain. We
then construct a global-local consistent classifier to classify SAM masks,
which reveals the intrinsic structure of high-quality embeddings produced by
DBA-CLIP and demonstrates robustness against noisy pseudo-labels. Extensive
experiments validate the efficiency and effectiveness of our method, and we
achieve state-of-the-art (SOTA) or competitive performance across various
datasets and supervision types.

本研究提出了一种无需调参的基于 CLIP 的语义分割框架，可以普遍适用于各种类型的监督。通过利用 CLIP 的零样本分类能力生成伪标签或进行开放词汇量的分割，我们解决了蒙版与 CLIP 文本嵌入之间的偏差问题，提出了一种判别偏差对齐的 CLIP 方法以获得更好的性能。我们构建了一个全局 - 局部一致性的分类器来对 SAM 蒙版进行分类，揭示了 DBA-CLIP 产生的高质量嵌入的内在结构，并展示了对噪声伪标签的鲁棒性。大量实验证实了我们方法的效率和有效性，并在各种数据集和监督类型上取得了最先进或具有竞争力的性能。