While tokenized graph Transformers have demonstrated strong performance in
node classification tasks, their reliance on a limited subset of nodes with
high similarity scores for constructing token sequences overlooks valuable
information from other nodes, hindering their ability to fully harness graph
information for learning optimal node representations. To address this
limitation, we propose a novel graph Transformer called GCFormer. Unlike
previous approaches, GCFormer develops a hybrid token generator to create two
types of token sequences, positive and negative, to capture diverse graph
information. And a tailored Transformer-based backbone is adopted to learn
meaningful node representations from these generated token sequences.
Additionally, GCFormer introduces contrastive learning to extract valuable
information from both positive and negative token sequences, enhancing the
quality of learned node representations. Extensive experimental results across
various datasets, including homophily and heterophily graphs, demonstrate the
superiority of GCFormer in node classification, when compared to representative
graph neural networks (GNNs) and graph Transformers.

提出了一种名为 GCFormer 的新型图形变压器，该模型使用混合令牌生成器来捕捉多样性图形信息的两种类型的令牌序列，并采用定制的基于变压器的骨干结构从这些生成的令牌序列中学习有意义的节点表示。并且，GCFormer 引入了对比学习来从正负令牌序列中提取有价值的信息，提高了学习到的节点表示的质量。在各种数据集上进行的大量实验结果表明，与代表性的图神经网络（GNNs）和图形变压器相比，GCFormer 在节点分类方面具有优势。

利用对比学习提升令牌化图变换器中节点表示

Leveraging Contrastive Learning for Enhanced Node Representations in  Tokenized Graph Transformers

Transformer-based models have emerged as promising tools for time series
forecasting.
However, these model cannot make accurate prediction for long input time
series. On the one hand, they failed to capture global dependencies within time
series data. On the other hand, the long input sequence usually leads to large
model size and high time complexity.
To address these limitations, we present GCformer, which combines a
structured global convolutional branch for processing long input sequences with
a local Transformer-based branch for capturing short, recent signals. A
cohesive framework for a global convolution kernel has been introduced,
utilizing three distinct parameterization methods. The selected structured
convolutional kernel in the global branch has been specifically crafted with
sublinear complexity, thereby allowing for the efficient and effective
processing of lengthy and noisy input signals. Empirical studies on six
benchmark datasets demonstrate that GCformer outperforms state-of-the-art
methods, reducing MSE error in multivariate time series benchmarks by 4.38% and
model parameters by 61.92%. In particular, the global convolutional branch can
serve as a plug-in block to enhance the performance of other models, with an
average improvement of 31.93\%, including various recently published
Transformer-based models. Our code is publicly available at
this https URL

GCformer 能在处理长输入序列的同时，利用结构化全局卷积分支处理全局依赖关系并利用本地 Transformer 分支捕获近期的短时信号，该方法在六个基准数据集上实验表明 GCformer 的表现优于现有方法，尤其其全局卷积分支能够增强其他模型的性能，平均改进 31.93％。