Vision Transformers have shown great potential in computer vision tasks. Most
recent works have focused on elaborating the spatial token mixer for
performance gains. However, we observe that a well-designed general
architecture can significantly improve the performance of the entire backbone,
regardless of which spatial token mixer is equipped. In this paper, we propose
UniNeXt, an improved general architecture for the vision backbone. To verify
its effectiveness, we instantiate the spatial token mixer with various typical
and modern designs, including both convolution and attention modules. Compared
with the architecture in which they are first proposed, our UniNeXt
architecture can steadily boost the performance of all the spatial token
mixers, and narrows the performance gap among them. Surprisingly, our UniNeXt
equipped with naive local window attention even outperforms the previous
state-of-the-art. Interestingly, the ranking of these spatial token mixers also
changes under our UniNeXt, suggesting that an excellent spatial token mixer may
be stifled due to a suboptimal general architecture, which further shows the
importance of the study on the general architecture of vision backbone. All
models and codes will be publicly available.

本研究提出了 UniNeXt，对计算机视觉中的通用骨干架构进行了改进，这提高了所有空间令牌混合器的性能，并缩小了它们之间的性能差距，甚至超越了之前的最优解，它还表明了对通用骨干架构的研究的重要性。

UniNeXt：探索一种统一的视觉识别架构

UniNeXt: Exploring A Unified Architecture for Vision Recognition

All instance perception tasks aim at finding certain objects specified by
some queries such as category names, language expressions, and target
annotations, but this complete field has been split into multiple independent
subtasks. In this work, we present a universal instance perception model of the
next generation, termed UNINEXT. UNINEXT reformulates diverse instance
perception tasks into a unified object discovery and retrieval paradigm and can
flexibly perceive different types of objects by simply changing the input
prompts. This unified formulation brings the following benefits: (1) enormous
data from different tasks and label vocabularies can be exploited for jointly
training general instance-level representations, which is especially beneficial
for tasks lacking in training data. (2) the unified model is
parameter-efficient and can save redundant computation when handling multiple
tasks simultaneously. UNINEXT shows superior performance on 20 challenging
benchmarks from 10 instance-level tasks including classical image-level tasks
(object detection and instance segmentation), vision-and-language tasks
(referring expression comprehension and segmentation), and six video-level
object tracking tasks. Code is available at
this https URL

本文提出了一个称之为 UNINEXT 的通用实例感知模型，它将各种实例感知任务重新制定为统一的对象发现和检索范式，并且可以灵活地通过更改输入提示来感知不同类型的对象。这个统一的公式带来了以下好处：(1) 可以利用来自不同任务和标签词汇的大量数据进行联合训练，这对于缺乏训练数据的任务尤其有益。(2) 这个统一的模型是参数高效的，能够在处理多个任务时节省冗余计算。在 20 个具有挑战性的基准测试中，包括经典的图像级任务 (对象检测和实例分割)、视觉语言任务 (指称理解和分割) 和六个视频级对象跟踪任务，UNINEXT 表现出优异的性能。