The three existing dominant network families, i.e., CNNs, Transformers, and
MLPs, differ from each other mainly in the ways of fusing spatial contextual
information, leaving designing more effective token-mixing mechanisms at the
core of backbone architecture development. In this work, we propose an
innovative token-mixer, dubbed Active Token Mixer (ATM), to actively
incorporate flexible contextual information distributed across different
channels from other tokens into the given query token. This fundamental
operator actively predicts where to capture useful contexts and learns how to
fuse the captured contexts with the query token at channel level. In this way,
the spatial range of token-mixing can be expanded to a global scope with
limited computational complexity, where the way of token-mixing is reformed. We
take ATM as the primary operator and assemble ATMs into a cascade architecture,
dubbed ATMNet. Extensive experiments demonstrate that ATMNet is generally
applicable and comprehensively surpasses different families of SOTA vision
backbones by a clear margin on a broad range of vision tasks, including visual
recognition and dense prediction tasks. Code is available at
this https URL

本文提出 ATMNet 模型，以 Active Token Mixer 为基础操作符，通过主动预测有用上下文的位置，并学习如何在通道级别上将这些上下文与 query token 融合，从而扩展了 token-mixing 的空间范围，提高了模型的有效性和可扩展性，实验证明 ATMNet 在各种视觉任务中均优于现有的 CNNs，Transformers 和 MLPs 等传统模型。