Graph neural networks are powerful architectures for structured datasets.
However, current methods struggle to represent long-range dependencies. Scaling
the depth or width of GNNs is insufficient to broaden receptive fields as
larger GNNs encounter optimization instabilities such as vanishing gradients
and representation oversmoothing, while pooling-based approaches have yet to
become as universally useful as in computer vision. In this work, we propose
the use of Transformer-based self-attention to learn long-range pairwise
relationships, with a novel "readout" mechanism to obtain a global graph
embedding. Inspired by recent computer vision results that find
position-invariant attention performant in learning long-range relationships,
our method, which we call GraphTrans, applies a permutation-invariant
Transformer module after a standard GNN module. This simple architecture leads
to state-of-the-art results on several graph classification tasks,
outperforming methods that explicitly encode graph structure. Our results
suggest that purely-learning-based approaches without graph structure may be
suitable for learning high-level, long-range relationships on graphs. Code for
GraphTrans is available at this https URL

本篇论文提出了一种名为 GraphTrans 的算法，它将 Transformer 自注意力模块和标准的 GNN 模块相结合，并应用置换不变性的 Transformer 模块，从而解决了当前方法在表示长程依赖关系方面的不足。该算法在多个图分类任务上表现出色，说明纯粹的基于学习的方法可能适用于在图上学习高水平、长程的关系。