Recently, linear complexity sequence modeling networks have achieved modeling
capabilities similar to Vision Transformers on a variety of computer vision
tasks, while using fewer FLOPs and less memory. However, their advantage in
terms of actual runtime speed is not significant. To address this issue, we
introduce Gated Linear Attention (GLA) for vision, leveraging its superior
hardware-awareness and efficiency. We propose direction-wise gating to capture
1D global context through bidirectional modeling and a 2D gating locality
injection to adaptively inject 2D local details into 1D global context. Our
hardware-aware implementation further merges forward and backward scanning into
a single kernel, enhancing parallelism and reducing memory cost and latency.
The proposed model, ViG, offers a favorable trade-off in accuracy, parameters,
and FLOPs on ImageNet and downstream tasks, outperforming popular Transformer
and CNN-based models. Notably, ViG-S matches DeiT-B's accuracy while using only
27% of the parameters and 20% of the FLOPs, running 2$\times$ faster on
$224\times224$ images. At $1024\times1024$ resolution, ViG-T uses 5.2$\times$
fewer FLOPs, saves 90% GPU memory, runs 4.8$\times$ faster, and achieves 20.7%
higher top-1 accuracy than DeiT-T. These results position ViG as an efficient
and scalable solution for visual representation learning. Code is available at
https://github.com/hustvl/ViG.

在计算机视觉任务中，线性复杂度序列建模网络达到了与 Vision Transformers 相似的建模能力，同时使用更少的 FLOPs 和内存。然而，它们在实际运行时速度方面的优势并不显著。为了解决这个问题，我们引入了适用于视觉的门控线性注意力（GLA），利用它的硬件感知和高效性。我们提出了方向性门控，通过双向建模来捕捉一维全局上下文，并通过二维门控局部注入来自适应地将二维局部细节注入一维全局上下文。我们的硬件感知实现将正向扫描和反向扫描合并为单个核心，增强了并行性，减少了内存成本和延迟。所提出的模型 ViG 在 ImageNet 和下游任务中在准确性、参数和 FLOPs 上取得了有利的权衡，优于流行的 Transformer 和基于 CNN 的模型。值得注意的是，ViG-S 仅使用参数的 27% 和 FLOPs 的 20%，在 224x224 大小的图像上运行速度比 DeiT-B 快 2 倍。在 1024x1024 分辨率下，ViG-T 使用的 FLOPs 比 DeiT-T 少了 5.2 倍，节省了 90% 的 GPU 内存，运行速度快了 4.8 倍，并且相比 DeiT-T 的 top-1 准确率高出 20.7%。这些结果使 ViG 成为一种高效且可扩展的视觉表示学习解决方案。代码可在 https://github.com/hustvl/ViG 找到。