A powerful architecture for universal segmentation relies on transformers
that encode multi-scale image features and decode object queries into mask
predictions. With efficiency being a high priority for scaling such models, we
observed that the state-of-the-art method Mask2Former uses ~50% of its compute
only on the transformer encoder. This is due to the retention of a full-length
token-level representation of all backbone feature scales at each encoder
layer. With this observation, we propose a strategy termed PROgressive Token
Length SCALing for Efficient transformer encoders (PRO-SCALE) that can be
plugged-in to the Mask2Former-style segmentation architectures to significantly
reduce the computational cost. The underlying principle of PRO-SCALE is:
progressively scale the length of the tokens with the layers of the encoder.
This allows PRO-SCALE to reduce computations by a large margin with minimal
sacrifice in performance (~52% GFLOPs reduction with no drop in performance on
COCO dataset). We validate our framework on multiple public benchmarks.

提出了一种名为 PRO-SCALE 的策略，用于可插拔于 Mask2Former 式分割架构中，通过逐层渐进地缩短令牌长度，显著降低计算成本（在 COCO 数据集上，计算量减少了约 52%，而性能没有下降）的同时确保高效的转换器编码器。

用于高效通用分段的 Transformer 编码器中的渐进式令牌长度缩放

Progressive Token Length Scaling in Transformer Encoders for Efficient  Universal Segmentation

This paper aims to achieve universal segmentation of arbitrary semantic
level. Despite significant progress in recent years, specialist segmentation
approaches are limited to specific tasks and data distribution. Retraining a
new model for adaptation to new scenarios or settings takes expensive
computation and time cost, which raises the demand for versatile and universal
segmentation model that can cater to various granularity. Although some
attempts have been made for unifying different segmentation tasks or
generalization to various scenarios, limitations in the definition of paradigms
and input-output spaces make it difficult for them to achieve accurate
understanding of content at arbitrary granularity. To this end, we present
UniLSeg, a universal segmentation model that can perform segmentation at any
semantic level with the guidance of language instructions. For training
UniLSeg, we reorganize a group of tasks from original diverse distributions
into a unified data format, where images with texts describing segmentation
targets as input and corresponding masks are output. Combined with a automatic
annotation engine for utilizing numerous unlabeled data, UniLSeg achieves
excellent performance on various tasks and settings, surpassing both specialist
and unified segmentation models.

实现任意语义级别的通用分割模型 UniLSeg，通过语言指导实现各种任务和设置的优秀性能。

用语言指导进行任意粒度的通用分割

Universal Segmentation at Arbitrary Granularity with Language  Instruction

A major enduring focus of clinical workflows is disease analytics and
diagnosis, leading to medical imaging datasets where the modalities and
annotations are strongly tied to specific clinical objectives. To date,
building task-specific segmentation models is intuitive yet a restrictive
approach, lacking insights gained from widespread imaging cohorts. Inspired by
the training of medical residents, we explore universal medical image
segmentation, whose goal is to learn from diverse medical imaging sources
covering a range of clinical targets, body regions, and image modalities.
Following this paradigm, we propose Hermes, a context prior learning approach
that addresses the challenges related to the heterogeneity on data, modality,
and annotations in the proposed universal paradigm. In a collection of seven
diverse datasets, we demonstrate the appealing merits of the universal paradigm
over the traditional task-specific training paradigm. By leveraging the synergy
among various tasks, Hermes shows superior performance and model scalability.
Our in-depth investigation on two additional datasets reveals Hermes' strong
capabilities for transfer learning, incremental learning, and generalization to
different downstream tasks. The code is available:
this https URL

提出了一种基于上下文先验学习的通用医学图像分割方法 Hermes，适用于不同临床目标、身体部位和图像模态，并在七个数据集上证明了其性能优异，并且具有传输学习和概括到不同下游任务的强大能力。