While local-window self-attention performs notably in vision tasks, it
suffers from limited receptive field and weak modeling capability issues. This
is mainly because it performs self-attention within non-overlapped windows and
shares weights on the channel dimension. We propose MixFormer to find a
solution. First, we combine local-window self-attention with depth-wise
convolution in a parallel design, modeling cross-window connections to enlarge
the receptive fields. Second, we propose bi-directional interactions across
branches to provide complementary clues in the channel and spatial dimensions.
These two designs are integrated to achieve efficient feature mixing among
windows and dimensions. Our MixFormer provides competitive results on image
classification with EfficientNet and shows better results than RegNet and Swin
Transformer. Performance in downstream tasks outperforms its alternatives by
significant margins with less computational costs in 5 dense prediction tasks
on MS COCO, ADE20k, and LVIS. Code is available at
https://github.com/PaddlePaddle/PaddleClas.

本论文提出了一种名为 MixFormer 的方法，将局部窗口自注意力与深度可分卷积相结合，跨窗口连接建模以扩大感受野，并在通道和空间维度上提供互补线索，从而实现更好的特征混合。在图像分类方面，MixFormer 相比于 RegNet 和 Swin Transformer 表现更好。在 MS COCO，ADE20k 和 LVIS 的 5 项密集预测任务中，其下游任务的性能也明显优于其他替代方案。

MixFormer: 跨窗口与跨维度混合特征

MixFormer: Mixing Features across Windows and Dimensions

We present a High-Resolution Transformer (HRFormer) that learns
high-resolution representations for dense prediction tasks, in contrast to the
original Vision Transformer that produces low-resolution representations and
has high memory and computational cost. We take advantage of the
multi-resolution parallel design introduced in high-resolution convolutional
networks (HRNet), along with local-window self-attention that performs
self-attention over small non-overlapping image windows, for improving the
memory and computation efficiency. In addition, we introduce a convolution into
the FFN to exchange information across the disconnected image windows. We
demonstrate the effectiveness of the High-Resolution Transformer on both human
pose estimation and semantic segmentation tasks, e.g., HRFormer outperforms
Swin transformer by $1.3$ AP on COCO pose estimation with $50\%$ fewer
parameters and $30\%$ fewer FLOPs. Code is available at:
this https URL

本研究提出了一种名为 High-Resolution Transformer 的方法，用于学习密集预测任务中的高分辨率表示， 并取得了在人体姿态估计和语义分割任务中较好的结果。