Semantic segmentation is a crucial task in computer vision, where each pixel
in an image is classified into a category. However, traditional methods face
significant challenges, including the need for pixel-level annotations and
extensive training. Furthermore, because supervised learning uses a limited set
of predefined categories, models typically struggle with rare classes and
cannot recognize new ones. Unsupervised and open-vocabulary segmentation,
proposed to tackle these issues, faces challenges, including the inability to
assign specific class labels to clusters and the necessity of user-provided
text queries for guidance. In this context, we propose a novel approach, TAG
which achieves Training, Annotation, and Guidance-free open-vocabulary semantic
segmentation. TAG utilizes pre-trained models such as CLIP and DINO to segment
images into meaningful categories without additional training or dense
annotations. It retrieves class labels from an external database, providing
flexibility to adapt to new scenarios. Our TAG achieves state-of-the-art
results on PascalVOC, PascalContext and ADE20K for open-vocabulary segmentation
without given class names, i.e. improvement of +15.3 mIoU on PascalVOC. All
code and data will be released at this https URL

TAG 是一种新颖的方法，用于在计算机视觉中进行无需训练、注释和指导的开放词汇语义分割，通过利用预训练模型以及来自外部数据库的类别标签检索，达到了在 PascalVOC、PascalContext 和 ADE20K 数据集上的开放词汇分割的最新成果。

无需指导的开放词汇语义分割

TAG: Guidance-free Open-Vocabulary Semantic Segmentation

We present a graph-based Tree Adjoining Grammar (TAG) parser that uses
BiLSTMs, highway connections, and character-level CNNs. Our best end-to-end
parser, which jointly performs supertagging, POS tagging, and parsing,
outperforms the previously reported best results by more than 2.2 LAS and UAS
points. The graph-based parsing architecture allows for global inference and
rich feature representations for TAG parsing, alleviating the fundamental
trade-off between transition-based and graph-based parsing systems. We also
demonstrate that the proposed parser achieves state-of-the-art performance in
the downstream tasks of Parsing Evaluation using Textual Entailments (PETE) and
Unbounded Dependency Recovery. This provides further support for the claim that
TAG is a viable formalism for problems that require rich structural analysis of
sentences.

本文介绍了基于图形的 Tree Adjoining Grammar (TAG) 解析器，并使用 BiLSTMs、Highway 连接和字符级 CNNs 来实现，该解析器在全局推理和特征表示方面具有先进性，在短语结构随从词性标注和解析方面性能显著优于以往的结果，并且在解析评估和无界依赖恢复方面实现了最先进的性能。