In recent years, Transformers have achieved remarkable progress in computer
vision tasks. However, their global modeling often comes with substantial
computational overhead, in stark contrast to the human eye's efficient
information processing. Inspired by the human eye's sparse scanning mechanism,
we propose a \textbf{S}parse \textbf{S}can \textbf{S}elf-\textbf{A}ttention
mechanism ($\rm{S}^3\rm{A}$). This mechanism predefines a series of Anchors of
Interest for each token and employs local attention to efficiently model the
spatial information around these anchors, avoiding redundant global modeling
and excessive focus on local information. This approach mirrors the human eye's
functionality and significantly reduces the computational load of vision
models. Building on $\rm{S}^3\rm{A}$, we introduce the \textbf{S}parse
\textbf{S}can \textbf{Vi}sion \textbf{T}ransformer (SSViT). Extensive
experiments demonstrate the outstanding performance of SSViT across a variety
of tasks. Specifically, on ImageNet classification, without additional
supervision or training data, SSViT achieves top-1 accuracies of
\textbf{84.4\%/85.7\%} with \textbf{4.4G/18.2G} FLOPs. SSViT also excels in
downstream tasks such as object detection, instance segmentation, and semantic
segmentation. Its robustness is further validated across diverse datasets. Code
will be available at https://github.com/qhfan/SSViT.

利用人眼的稀疏扫描机制，通过引入稀疏扫描自注意机制（S^3A）和稀疏扫描视觉 Transformer（SSViT），有效降低计算负荷，达到在计算机视觉任务中出色的性能表现。