This work presents ParFormer as an enhanced transformer architecture that
allows the incorporation of different token mixers into a single stage, hence
improving feature extraction capabilities. Integrating both local and global
data allows for precise representation of short- and long-range spatial
relationships without the need for computationally intensive methods such as
shifting windows. Along with the parallel token mixer encoder, We offer the
Convolutional Attention Patch Embedding (CAPE) as an enhancement of standard
patch embedding to improve token mixer extraction with a convolutional
attention module. Our comprehensive evaluation demonstrates that our ParFormer
outperforms CNN-based and state-of-the-art transformer-based architectures in
image classification and several complex tasks such as object recognition. The
proposed CAPE has been demonstrated to benefit the overall MetaFormer
architecture, even while utilizing the Identity Mapping Token Mixer, resulting
in a 0.5\% increase in accuracy. The ParFormer models outperformed ConvNeXt and
Swin Transformer for the pure convolution and transformer model in accuracy.
Furthermore, our model surpasses the current leading hybrid transformer by
reaching competitive Top-1 scores in the ImageNet-1K classification test.
Specifically, our model variants with 11M, 23M, and 34M parameters achieve
scores of 80.4\%, 82.1\%, and 83.1\%, respectively. Code:
this https URL

该研究提出了 ParFormer 作为一种改进的 transformer 体系结构，通过将不同的 token mixer 集成到一个阶段中，从而提高特征提取能力。与传统的窗口移动方法不同，结合本地和全局数据可以精确表示短程和长程空间关系。通过并行的 token mixer 编码器以及卷积注意力模块，我们提出了 Enhance Patch Embedding for Token Mixer (CEPTM) 来改善 token mixer 的提取能力。全面的评估结果表明，我们的 ParFormer 在图像分类和目标识别等多个复杂任务中表现优于基于 CNN 和最先进的 transformer 的架构。提出的 CEPTM 在使用 Identity Mapping Token Mixer 时也对整体 MetaFormer 体系结构有所改善，提高了 0.5％的准确性。ParFormer 模型在准确性方面优于纯卷积模型和 transformer 模型的 ConvNeXt 和 Swin Transformer。此外，我们的模型在 ImageNet-1K 分类测试中取得了领先的混合 transformer 模型达到了有竞争力的 Top-1 得分。具体来说，我们的模型变种，参数为 11M，23M 和 34M 时，分别达到了 80.4％，82.1％和 83.1％的得分。