Segmenting unknown or anomalous object instances is a critical task in
autonomous driving applications, and it is approached traditionally as a
per-pixel classification problem. However, reasoning individually about each
pixel without considering their contextual semantics results in high
uncertainty around the objects' boundaries and numerous false positives. We
propose a paradigm change by shifting from a per-pixel classification to a mask
classification. Our mask-based method, Mask2Anomaly, demonstrates the
feasibility of integrating a mask-classification architecture to jointly
address anomaly segmentation, open-set semantic segmentation, and open-set
panoptic segmentation. Mask2Anomaly includes several technical novelties that
are designed to improve the detection of anomalies/unknown objects: i) a global
masked attention module to focus individually on the foreground and background
regions; ii) a mask contrastive learning that maximizes the margin between an
anomaly and known classes; iii) a mask refinement solution to reduce false
positives; and iv) a novel approach to mine unknown instances based on the
mask-architecture properties. By comprehensive qualitative and qualitative
evaluation, we show Mask2Anomaly achieves new state-of-the-art results across
the benchmarks of anomaly segmentation, open-set semantic segmentation, and
open-set panoptic segmentation.

将传统的按像素分类的方法转变为基于掩码分类的方法，提出了 Mask2Anomaly 方法来解决异常分割、开放集语义分割和开放集全景分割问题，并引入了全局掩码注意力模块、掩码对比学习、掩码细化解决方案和基于掩码架构特性的未知实例挖掘。经过全面的定性和定量评估，我们展示了 Mask2Anomaly 在异常分割、开放集语义分割和开放集全景分割基准中取得了最新的最佳结果。

Mask2Anomaly：面具变换器用于通用开放集分割

Mask2Anomaly: Mask Transformer for Universal Open-set Segmentation

We extend panoptic segmentation to the open-world and introduce an open-set
panoptic segmentation (OPS) task. This task requires performing panoptic
segmentation for not only known classes but also unknown ones that have not
been acknowledged during training. We investigate the practical challenges of
the task and construct a benchmark on top of an existing dataset, COCO. In
addition, we propose a novel exemplar-based open-set panoptic segmentation
network (EOPSN) inspired by exemplar theory. Our approach identifies a new
class based on exemplars, which are identified by clustering and employed as
pseudo-ground-truths. The size of each class increases by mining new exemplars
based on the similarities to the existing ones associated with the class. We
evaluate EOPSN on the proposed benchmark and demonstrate the effectiveness of
our proposals. The primary goal of our work is to draw the attention of the
community to the recognition in the open-world scenarios. The implementation of
our algorithm is available on the project webpage:
this https URL

本研究提出了一种基于样本的开集全景分割网络（EOPSN），针对未知类开启了一项开放式全景分割（OPS）任务，并在 COCO 数据集上构建了标准测试。通过对样本的聚类和伪基准的应用，我们的方法识别了新类别并证明了其在开放式情景下的有效性。