Graph neural networks (GNNs) and variations of the message passing algorithm
are the predominant means for learning on graphs, largely due to their
flexibility, speed, and satisfactory performance. The design of powerful and
general purpose GNNs, however, requires significant research efforts and often
relies on handcrafted, carefully-chosen message passing operators. Motivated by
this, we propose a remarkably simple alternative for learning on graphs that
relies exclusively on attention. Graphs are represented as node or edge sets
and their connectivity is enforced by masking the attention weight matrix,
effectively creating custom attention patterns for each graph. Despite its
simplicity, masked attention for graphs (MAG) has state-of-the-art performance
on long-range tasks and outperforms strong message passing baselines and much
more involved attention-based methods on over 55 node and graph-level tasks. We
also show significantly better transfer learning capabilities compared to GNNs
and comparable or better time and memory scaling. MAG has sub-linear memory
scaling in the number of nodes or edges, enabling learning on dense graphs and
future-proofing the approach.

提出了一种基于注意力机制的学习图表的简单替代方法，被称为图形掩码注意力（MAG），在长距离任务上具有最先进的性能，并在超过 55 个节点和图级任务中优于强大的信息传递基线和复杂的注意力方法。与图神经网络相比，显示了显着更好的迁移学习能力，并具有与节点或边数目的次线性内存缩放相比较的相当或更好的时间和内存缩放性能。