The paradigm of Transformers using the self-attention mechanism has
manifested its advantage in learning graph-structured data. Yet, Graph
Transformers are capable of modeling full range dependencies but are often
deficient in extracting information from locality. A common practice is to
utilize Message Passing Neural Networks (MPNNs) as an auxiliary to capture
local information, which however are still inadequate for comprehending
substructures. In this paper, we present a purely attention-based architecture,
namely Focal and Full-Range Graph Transformer (FFGT), which can mitigate the
loss of local information in learning global correlations. The core component
of FFGT is a new mechanism of compound attention, which combines the
conventional full-range attention with K-hop focal attention on ego-nets to
aggregate both global and local information. Beyond the scope of canonical
Transformers, the FFGT has the merit of being more substructure-aware. Our
approach enhances the performance of existing Graph Transformers on various
open datasets, while achieves compatible SOTA performance on several Long-Range
Graph Benchmark (LRGB) datasets even with a vanilla transformer. We further
examine influential factors on the optimal focal length of attention via
introducing a novel synthetic dataset based on SBM-PATTERN.

基于纯注意力机制的 FFGT 图形 Transformer 可以缓解学习全局相关性时丢失本地信息的问题，并通过将全融注意力与邻域注意力相结合来聚合全局和本地信息，从而提高了现有图形 Transformer 在各种开放数据集上的性能。