In this paper, we present Vision Permutator, a conceptually simple and data
efficient MLP-like architecture for visual recognition. By realizing the
importance of the positional information carried by 2D feature representations,
unlike recent MLP-like models that encode the spatial information along the
flattened spatial dimensions, Vision Permutator separately encodes the feature
representations along the height and width dimensions with linear projections.
This allows Vision Permutator to capture long-range dependencies along one
spatial direction and meanwhile preserve precise positional information along
the other direction. The resulting position-sensitive outputs are then
aggregated in a mutually complementing manner to form expressive
representations of the objects of interest. We show that our Vision Permutators
are formidable competitors to convolutional neural networks (CNNs) and vision
transformers. Without the dependence on spatial convolutions or attention
mechanisms, Vision Permutator achieves 81.5% top-1 accuracy on ImageNet without
extra large-scale training data (e.g., ImageNet-22k) using only 25M learnable
parameters, which is much better than most CNNs and vision transformers under
the same model size constraint. When scaling up to 88M, it attains 83.2% top-1
accuracy. We hope this work could encourage research on rethinking the way of
encoding spatial information and facilitate the development of MLP-like models.
Code is available at this https URL

本文提出 Vision Permutator，这是一种类似于 MLP 的架构，用于视觉识别。通过在高度和宽度维度上分别编码特征表示，Vision Permutator 能够捕捉沿一个空间方向的长程依赖关系，同时保留另一个方向上的精确位置信息。在 ImageNet 数据集上，使用仅有 25M 可学习参数的 Vision Permutator 在不使用大规模训练数据（如 ImageNet-22k）的情况下可实现 81.5％的 top-1 准确性，比大多数 CNN 和视觉变换器都要好。