We present CSWin Transformer, an efficient and effective Transformer-based
backbone for general-purpose vision tasks. A challenging issue in Transformer
design is that global self-attention is very expensive to compute whereas local
self-attention often limits the field of interactions of each token. To address
this issue, we develop the Cross-Shaped Window self-attention mechanism for
computing self-attention in the horizontal and vertical stripes in parallel
that form a cross-shaped window, with each stripe obtained by splitting the
input feature into stripes of equal width. We provide a mathematical analysis
of the effect of the stripe width and vary the stripe width for different
layers of the Transformer network which achieves strong modeling capability
while limiting the computation cost. We also introduce Locally-enhanced
Positional Encoding (LePE), which handles the local positional information
better than existing encoding schemes. LePE naturally supports arbitrary input
resolutions, and is thus especially effective and friendly for downstream
tasks. Incorporated with these designs and a hierarchical structure, CSWin
Transformer demonstrates competitive performance on common vision tasks.
Specifically, it achieves 85.4\% Top-1 accuracy on ImageNet-1K without any
extra training data or label, 53.9 box AP and 46.4 mask AP on the COCO
detection task, and 52.2 mIOU on the ADE20K semantic segmentation task,
surpassing previous state-of-the-art Swin Transformer backbone by +1.2, +2.0,
+1.4, and +2.0 respectively under the similar FLOPs setting. By further
pretraining on the larger dataset ImageNet-21K, we achieve 87.5% Top-1 accuracy
on ImageNet-1K and high segmentation performance on ADE20K with 55.7 mIoU. The
code and models are available at
this https URL

CSWin Transformer 是一种有效的基于 Transformer 网络的视觉任务骨干，并通过使用 Cross-Shaped Window 自注意机制和局部增强位置编码等设计来提高建模性能，在常见的视觉任务中展现出了竞争力。