Network architecture plays a key role in the deep learning-based computer
vision system. The widely-used convolutional neural network and transformer
treat the image as a grid or sequence structure, which is not flexible to
capture irregular and complex objects. In this paper, we propose to represent
the image as a graph structure and introduce a new Vision GNN (ViG)
architecture to extract graph-level feature for visual tasks. We first split
the image to a number of patches which are viewed as nodes, and construct a
graph by connecting the nearest neighbors. Based on the graph representation of
images, we build our ViG model to transform and exchange information among all
the nodes. ViG consists of two basic modules: Grapher module with graph
convolution for aggregating and updating graph information, and FFN module with
two linear layers for node feature transformation. Both isotropic and pyramid
architectures of ViG are built with different model sizes. Extensive
experiments on image recognition and object detection tasks demonstrate the
superiority of our ViG architecture. We hope this pioneering study of GNN on
general visual tasks will provide useful inspiration and experience for future
research. The PyTorch code is available at
this https URL and the MindSpore code is
available at this https URL

本文提出了将图像表示为图结构，并介绍了一种新的 Vision GNN（ViG）体系结构，用于提取视觉任务的图级特征。ViG 由 Grapher 模块和 FFN 模块组成，可用于图形聚合、更新图形信息以及节点特征变换。该体系结构在图像识别和目标检测任务方面表现出优异性能，并希望将来的研究能够从中获得启发。