Traditionally, convolutional neural networks (CNN) and vision transformers
(ViT) have dominated computer vision. However, recently proposed vision graph
neural networks (ViG) provide a new avenue for exploration. Unfortunately, for
mobile applications, ViGs are computationally expensive due to the overhead of
representing images as graph structures. In this work, we propose a new
graph-based sparse attention mechanism, Sparse Vision Graph Attention (SVGA),
that is designed for ViGs running on mobile devices. Additionally, we propose
the first hybrid CNN-GNN architecture for vision tasks on mobile devices,
MobileViG, which uses SVGA. Extensive experiments show that MobileViG beats
existing ViG models and existing mobile CNN and ViT architectures in terms of
accuracy and/or speed on image classification, object detection, and instance
segmentation tasks. Our fastest model, MobileViG-Ti, achieves 75.7% top-1
accuracy on ImageNet-1K with 0.78 ms inference latency on iPhone 13 Mini NPU
(compiled with CoreML), which is faster than MobileNetV2x1.4 (1.02 ms, 74.7%
top-1) and MobileNetV2x1.0 (0.81 ms, 71.8% top-1). Our largest model,
MobileViG-B obtains 82.6% top-1 accuracy with only 2.30 ms latency, which is
faster and more accurate than the similarly sized EfficientFormer-L3 model
(2.77 ms, 82.4%). Our work proves that well designed hybrid CNN-GNN
architectures can be a new avenue of exploration for designing models that are
extremely fast and accurate on mobile devices. Our code is publicly available
at this https URL

本文提出了一种新颖的基于图的稀疏注意力机制 Sparse Vision Graph Attention（SVGA），并且设计了一种新的混合 CNN-GNN 体系结构 MobileViG，实验证明，MobileViG 在移动设备上进行图像分类、物体检测和实例分割任务上的精度和 / 或速度明显优于现有的 ViG 模型和现有移动 CNN 和 ViT 架构。