In recent years, the Transformer architecture has proven to be very
successful in sequence processing, but its application to other data
structures, such as graphs, has remained limited due to the difficulty of
properly defining positions. Here, we present the $\textit{Spectral Attention
Network}$ (SAN), which uses a learned positional encoding (LPE) that can take
advantage of the full Laplacian spectrum to learn the position of each node in
a given graph. This LPE is then added to the node features of the graph and
passed to a fully-connected Transformer. By leveraging the full spectrum of the
Laplacian, our model is theoretically powerful in distinguishing graphs, and
can better detect similar sub-structures from their resonance. Further, by
fully connecting the graph, the Transformer does not suffer from
over-squashing, an information bottleneck of most GNNs, and enables better
modeling of physical phenomenons such as heat transfer and electric
interaction. When tested empirically on a set of 4 standard datasets, our model
performs on par or better than state-of-the-art GNNs, and outperforms any
attention-based model by a wide margin, becoming the first fully-connected
architecture to perform well on graph benchmarks.

本文介绍了新的图神经网络 $	extit {Spectral Attention Network}$ (SAN)，通过学习节点位置编码 (LPE) 来处理图结构数据，并采用全连接 Transformer 作为模型框架，有效地解决了信息瓶颈问题和节点位置定义问题，从而在 4 个标准元数据集上表现出超过同类模型的性能。