Vision transformers have shown excellent performance in computer vision
tasks. As the computation cost of their self-attention mechanism is expensive,
recent works tried to replace the self-attention mechanism in vision
transformers with convolutional operations, which is more efficient with
built-in inductive bias. However, these efforts either ignore multi-level
features or lack dynamic prosperity, leading to sub-optimal performance. In
this paper, we propose a Dynamic Multi-level Attention mechanism (DMA), which
captures different patterns of input images by multiple kernel sizes and
enables input-adaptive weights with a gating mechanism. Based on DMA, we
present an efficient backbone network named DMFormer. DMFormer adopts the
overall architecture of vision transformers, while replacing the self-attention
mechanism with our proposed DMA. Extensive experimental results on ImageNet-1K
and ADE20K datasets demonstrated that DMFormer achieves state-of-the-art
performance, which outperforms similar-sized vision transformers(ViTs) and
convolutional neural networks (CNNs).

本文提出了一种动态多级注意力机制 (DMA)，它通过多个卷积核大小捕捉输入图像的不同模式，并通过门控机制实现输入自适应权重，然后提出了一种名为 DMFormer 的有效骨干网络，该网络采用了 DMA 替代了视觉变换器中的自我关注机制。在 ImageNet-1K 和 ADE20K 数据集上的广泛实验结果表明，DMFormer 具有先进的性能，优于大小相似的视觉变压器 (ViTs) 和卷积神经网络 (CNNs)。