The successful application of semantic segmentation technology in the real
world has been among the most exciting achievements in the computer vision
community over the past decade. Although the long-tailed phenomenon has been
investigated in many fields, e.g., classification and object detection, it has
not received enough attention in semantic segmentation and has become a
non-negligible obstacle to applying semantic segmentation technology in
autonomous driving and virtual reality. Therefore, in this work, we focus on a
relatively under-explored task setting, long-tailed semantic segmentation
(LTSS). We first establish three representative datasets from different
aspects, i.e., scene, object, and human. We further propose a dual-metric
evaluation system and construct the LTSS benchmark to demonstrate the
performance of semantic segmentation methods and long-tailed solutions. We also
propose a transformer-based algorithm to improve LTSS, frequency-based matcher,
which solves the oversuppression problem by one-to-many matching and
automatically determines the number of matching queries for each class. Given
the comprehensiveness of this work and the importance of the issues revealed,
this work aims to promote the empirical study of semantic segmentation tasks.
Our datasets, codes, and models will be publicly available.

本文关注于长尾语义分割 (LTSS) 这一相对未经深入研究的任务设置，建立了三个代表性数据集，并提出了基于 Transformer 的改进算法，用于解决应用语义分割技术在自动驾驶和虚拟现实中遇到的长尾问题，并通过 LTSS 评估系统和基准进行性能展示，旨在推动对语义分割任务的实证研究。

基于频率匹配的长尾语义分割

Frequency-based Matcher for Long-tailed Semantic Segmentation

Conventional super-resolution methods suffer from two drawbacks: substantial
computational cost in upscaling an entire large image, and the introduction of
extraneous or potentially detrimental information for downstream computer
vision tasks during the refinement of the background. To solve these issues, we
propose a novel transformer-based algorithm, Selective Super-Resolution (SSR),
which partitions images into non-overlapping tiles, selects tiles of interest
at various scales with a pyramid architecture, and exclusively reconstructs
these selected tiles with deep features. Experimental results on three datasets
demonstrate the efficiency and robust performance of our approach for
super-resolution. Compared to the state-of-the-art methods, the FID score is
reduced from 26.78 to 10.41 with 40% reduction in computation cost for the
BDD100K dataset. The source code is available at
this https URL

通过将图像分割为不重叠的图块、使用金字塔结构选择感兴趣的图块，并利用深度特征仅重构这些选定的图块，我们提出了一种新颖的基于变压器的算法 Selective Super-Resolution（SSR），解决了常规超分辨率方法在放大整个大图像时的巨大计算成本和在背景细化过程中引入额外或可能有害信息的两个缺点。实验证明，我们的方法在超分辨率方面具备效率和稳健性，在 BDD100K 数据集上与最先进的方法相比，FID 评分从 26.78 降低到 10.41，计算成本减少了 40%。源代码可在此 https URL 获得。