While tokenized graph Transformers have demonstrated strong performance in
node classification tasks, their reliance on a limited subset of nodes with
high similarity scores for constructing token sequences overlooks valuable
information from other nodes, hindering their ability to fully harness graph
information for learning optimal node representations. To address this
limitation, we propose a novel graph Transformer called GCFormer. Unlike
previous approaches, GCFormer develops a hybrid token generator to create two
types of token sequences, positive and negative, to capture diverse graph
information. And a tailored Transformer-based backbone is adopted to learn
meaningful node representations from these generated token sequences.
Additionally, GCFormer introduces contrastive learning to extract valuable
information from both positive and negative token sequences, enhancing the
quality of learned node representations. Extensive experimental results across
various datasets, including homophily and heterophily graphs, demonstrate the
superiority of GCFormer in node classification, when compared to representative
graph neural networks (GNNs) and graph Transformers.

提出了一种名为 GCFormer 的新型图形变压器，该模型使用混合令牌生成器来捕捉多样性图形信息的两种类型的令牌序列，并采用定制的基于变压器的骨干结构从这些生成的令牌序列中学习有意义的节点表示。并且，GCFormer 引入了对比学习来从正负令牌序列中提取有价值的信息，提高了学习到的节点表示的质量。在各种数据集上进行的大量实验结果表明，与代表性的图神经网络（GNNs）和图形变压器相比，GCFormer 在节点分类方面具有优势。

利用对比学习提升令牌化图变换器中节点表示

Leveraging Contrastive Learning for Enhanced Node Representations in  Tokenized Graph Transformers

Graph neural networks (GNNs) have been widely investigated in the field of
semi-supervised graph machine learning. Most methods fail to exploit adequate
graph information when labeled data is limited, leading to the problem of
oversmoothing. To overcome this issue, we propose the Graph Alignment Neural
Network (GANN), a simple and effective graph neural architecture. A unique
learning algorithm with three alignment rules is proposed to thoroughly explore
hidden information for insufficient labels. Firstly, to better investigate
attribute specifics, we suggest the feature alignment rule to align the inner
product of both the attribute and embedding matrices. Secondly, to properly
utilize the higher-order neighbor information, we propose the cluster center
alignment rule, which involves aligning the inner product of the cluster center
matrix with the unit matrix. Finally, to get reliable prediction results with
few labels, we establish the minimum entropy alignment rule by lining up the
prediction probability matrix with its sharpened result. Extensive studies on
graph benchmark datasets demonstrate that GANN can achieve considerable
benefits in semi-supervised node classification and outperform state-of-the-art
competitors.

本文提出了一种简单而有效的 GNN 体系结构，名为 GANN，采用三个新型的图信息对齐规则，以解决标记数据有限导致 over-smoothing 的问题，并在半监督图节点分类任务中拥有优于竞争对手的表现。