While tokenized graph Transformers have demonstrated strong performance in node classification tasks, their reliance on a limited subset of nodes with high similarity scores for constructing token sequences overlooks valuable information from other nodes, hindering their ability to fully harness graph information for learning optimal node representations. To address this limitation, we propose a novel graph Transformer called GCFormer. Unlike previous approaches, GCFormer develops a hybrid token generator to create two types of token sequences, positive and negative, to capture diverse graph information. And a tailored Transformer-based backbone is adopted to learn meaningful node representations from these generated token sequences. Additionally, GCFormer introduces contrastive learning to extract valuable information from both positive and negative token sequences, enhancing the quality of learned node representations. Extensive experimental results across various datasets, including homophily and heterophily graphs, demonstrate the superiority of GCFormer in node classification, when compared to representative graph neural networks (GNNs) and graph Transformers.

提出了一种名为GCFormer的新型图形变压器，该模型使用混合令牌生成器来捕捉多样性图形信息的两种类型的令牌序列，并采用定制的基于变压器的骨干结构从这些生成的令牌序列中学习有意义的节点表示。并且，GCFormer引入了对比学习来从正负令牌序列中提取有价值的信息，提高了学习到的节点表示的质量。在各种数据集上进行的大量实验结果表明，与代表性的图神经网络（GNNs）和图形变压器相比，GCFormer在节点分类方面具有优势。

利用对比学习提升令牌化图变换器中节点表示