Transformers have sprung up in the field of computer vision. In this work, we
explore whether the core self-attention module in Transformer is the key to
achieving excellent performance in image recognition. To this end, we build an
attention-free network called sMLPNet based on the existing MLP-based vision
models. Specifically, we replace the MLP module in the token-mixing step with a
novel sparse MLP (sMLP) module. For 2D image tokens, sMLP applies 1D MLP along
the axial directions and the parameters are shared among rows or columns. By
sparse connection and weight sharing, sMLP module significantly reduces the
number of model parameters and computational complexity, avoiding the common
over-fitting problem that plagues the performance of MLP-like models. When only
trained on the ImageNet-1K dataset, the proposed sMLPNet achieves 81.9% top-1
accuracy with only 24M parameters, which is much better than most CNNs and
vision Transformers under the same model size constraint. When scaling up to
66M parameters, sMLPNet achieves 83.4% top-1 accuracy, which is on par with the
state-of-the-art Swin Transformer. The success of sMLPNet suggests that the
self-attention mechanism is not necessarily a silver bullet in computer vision.
The code and models are publicly available at
this https URL

本文研究了 Transformer 的核心自注意力机制在图像识别领域中扮演的关键角色。作者提出了一种名为 sMLPNet 的无注意力网络，其中使用了一种新的稀疏 MLP（sMLP）模块，避免了常见的过拟合问题，取得了优异的结果。