While local-window self-attention performs notably in vision tasks, it
suffers from limited receptive field and weak modeling capability issues. This
is mainly because it performs self-attention within non-overlapped windows and
shares weights on the channel dimension. We propose MixFormer to find a
solution. First, we combine local-window self-attention with depth-wise
convolution in a parallel design, modeling cross-window connections to enlarge
the receptive fields. Second, we propose bi-directional interactions across
branches to provide complementary clues in the channel and spatial dimensions.
These two designs are integrated to achieve efficient feature mixing among
windows and dimensions. Our MixFormer provides competitive results on image
classification with EfficientNet and shows better results than RegNet and Swin
Transformer. Performance in downstream tasks outperforms its alternatives by
significant margins with less computational costs in 5 dense prediction tasks
on MS COCO, ADE20k, and LVIS. Code is available at
https://github.com/PaddlePaddle/PaddleClas.

本论文提出了一种名为 MixFormer 的方法，将局部窗口自注意力与深度可分卷积相结合，跨窗口连接建模以扩大感受野，并在通道和空间维度上提供互补线索，从而实现更好的特征混合。在图像分类方面，MixFormer 相比于 RegNet 和 Swin Transformer 表现更好。在 MS COCO，ADE20k 和 LVIS 的 5 项密集预测任务中，其下游任务的性能也明显优于其他替代方案。