While tokenized graph Transformers have demonstrated strong performance in
node classification tasks, their reliance on a limited subset of nodes with
high similarity scores for constructing token sequences overlooks valuable
information from other nodes, hindering their ability to fully harness graph
information for learning optimal node representations. To address this
limitation, we propose a novel graph Transformer called GCFormer. Unlike
previous approaches, GCFormer develops a hybrid token generator to create two
types of token sequences, positive and negative, to capture diverse graph
information. And a tailored Transformer-based backbone is adopted to learn
meaningful node representations from these generated token sequences.
Additionally, GCFormer introduces contrastive learning to extract valuable
information from both positive and negative token sequences, enhancing the
quality of learned node representations. Extensive experimental results across
various datasets, including homophily and heterophily graphs, demonstrate the
superiority of GCFormer in node classification, when compared to representative
graph neural networks (GNNs) and graph Transformers.

提出了一种名为 GCFormer 的新型图形变压器，该模型使用混合令牌生成器来捕捉多样性图形信息的两种类型的令牌序列，并采用定制的基于变压器的骨干结构从这些生成的令牌序列中学习有意义的节点表示。并且，GCFormer 引入了对比学习来从正负令牌序列中提取有价值的信息，提高了学习到的节点表示的质量。在各种数据集上进行的大量实验结果表明，与代表性的图神经网络（GNNs）和图形变压器相比，GCFormer 在节点分类方面具有优势。