Segmenting unknown or anomalous object instances is a critical task in
autonomous driving applications, and it is approached traditionally as a
per-pixel classification problem. However, reasoning individually about each
pixel without considering their contextual semantics results in high
uncertainty around the objects' boundaries and numerous false positives. We
propose a paradigm change by shifting from a per-pixel classification to a mask
classification. Our mask-based method, Mask2Anomaly, demonstrates the
feasibility of integrating a mask-classification architecture to jointly
address anomaly segmentation, open-set semantic segmentation, and open-set
panoptic segmentation. Mask2Anomaly includes several technical novelties that
are designed to improve the detection of anomalies/unknown objects: i) a global
masked attention module to focus individually on the foreground and background
regions; ii) a mask contrastive learning that maximizes the margin between an
anomaly and known classes; iii) a mask refinement solution to reduce false
positives; and iv) a novel approach to mine unknown instances based on the
mask-architecture properties. By comprehensive qualitative and qualitative
evaluation, we show Mask2Anomaly achieves new state-of-the-art results across
the benchmarks of anomaly segmentation, open-set semantic segmentation, and
open-set panoptic segmentation.

将传统的按像素分类的方法转变为基于掩码分类的方法，提出了 Mask2Anomaly 方法来解决异常分割、开放集语义分割和开放集全景分割问题，并引入了全局掩码注意力模块、掩码对比学习、掩码细化解决方案和基于掩码架构特性的未知实例挖掘。经过全面的定性和定量评估，我们展示了 Mask2Anomaly 在异常分割、开放集语义分割和开放集全景分割基准中取得了最新的最佳结果。

Mask2Anomaly：面具变换器用于通用开放集分割

Mask2Anomaly: Mask Transformer for Universal Open-set Segmentation

In semantic segmentation, adapting a visual system to novel object categories
at inference time has always been both valuable and challenging. To enable such
generalization, existing methods rely on either providing several support
examples as visual cues or class names as textual cues. Through the development
is relatively optimistic, these two lines have been studied in isolation,
neglecting the complementary intrinsic of low-level visual and high-level
language information. In this paper, we define a unified setting termed as
open-set semantic segmentation (O3S), which aims to learn seen and unseen
semantics from both visual examples and textual names. Our pipeline extracts
multi-modal prototypes for segmentation task, by first single modal
self-enhancement and aggregation, then multi-modal complementary fusion. To be
specific, we aggregate visual features into several tokens as visual
prototypes, and enhance the class name with detailed descriptions for textual
prototype generation. The two modalities are then fused to generate multi-modal
prototypes for final segmentation. On both \pascal and \coco datasets, we
conduct extensive experiments to evaluate the framework effectiveness.
State-of-the-art results are achieved even on more detailed part-segmentation,
Pascal-Animals, by only training on coarse-grained datasets. Thorough ablation
studies are performed to dissect each component, both quantitatively and
qualitatively.

本文提出了一种称为开放式语义分割（O3S）的统一框架，在视觉示例和文本名称中学习已见和未见的语义。我们的流程通过首先进行单模自我增强和聚合，然后进行多模态补充融合来提取用于分割任务的多模态原型。