Pretraining plays a pivotal role in acquiring generalized knowledge from
large-scale data, achieving remarkable successes as evidenced by large models
in CV and NLP. However, progress in the graph domain remains limited due to
fundamental challenges such as feature heterogeneity and structural
heterogeneity. Recently, increasing efforts have been made to enhance node
feature quality with Large Language Models (LLMs) on text-attributed graphs
(TAGs), demonstrating superiority to traditional bag-of-words or word2vec
techniques. These high-quality node features reduce the previously critical
role of graph structure, resulting in a modest performance gap between Graph
Neural Networks (GNNs) and structure-agnostic Multi-Layer Perceptrons (MLPs).
Motivated by this, we introduce a feature-centric pretraining perspective by
treating graph structure as a prior and leveraging the rich, unified feature
space to learn refined interaction patterns that generalizes across graphs. Our
framework, Graph Sequence Pretraining with Transformer (GSPT), samples node
contexts through random walks and employs masked feature reconstruction to
capture pairwise proximity in the LLM-unified feature space using a standard
Transformer. By utilizing unified text representations rather than varying
structures, our framework achieves significantly better transferability among
graphs within the same domain. GSPT can be easily adapted to both node
classification and link prediction, demonstrating promising empirical success
on various datasets.

图序列预训练框架 GSPT 利用统一的文本表示，在图领域中取得了显著的可转移性和实证成功。

文本属性图上的纯 Transformer 预训练框架

A Pure Transformer Pretraining Framework on Text-attributed Graphs

The existing definitions of graph convolution, either from spatial or
spectral perspectives, are inflexible and not unified. Defining a general
convolution operator in the graph domain is challenging due to the lack of
canonical coordinates, the presence of irregular structures, and the properties
of graph symmetries. In this work, we propose a novel graph convolution
framework by parameterizing the kernels as continuous functions of
pseudo-coordinates derived via graph positional encoding. We name this
Continuous Kernel Graph Convolution (CKGConv). Theoretically, we demonstrate
that CKGConv is flexible and expressive. CKGConv encompasses many existing
graph convolutions, and exhibits the same expressiveness as graph transformers
in terms of distinguishing non-isomorphic graphs. Empirically, we show that
CKGConv-based Networks outperform existing graph convolutional networks and
perform comparably to the best graph transformers across a variety of graph
datasets.

我们提出了一种新颖的连续核图卷积（CKGConv）框架，通过将核函数参数化为通过图位置编码得到的伪坐标的连续函数，从而解决了在图域中定义通用卷积算子的挑战，该框架理论上证明了 CKGConv 的灵活性和表现力，并在实证中证明 CKGConv 网络在各种图数据集上优于现有的图卷积网络，并与最佳图转换器表现相当。

CKGConv: 连续核的通用图卷积

CKGConv: General Graph Convolution with Continuous Kernels

Out-of-distribution (OOD) generalization in the graph domain is challenging
due to complex distribution shifts and a lack of environmental contexts. Recent
methods attempt to enhance graph OOD generalization by generating flat
environments. However, such flat environments come with inherent limitations to
capture more complex data distributions. Considering the DrugOOD dataset, which
contains diverse training environments (e.g., scaffold, size, etc.), flat
contexts cannot sufficiently address its high heterogeneity. Thus, a new
challenge is posed to generate more semantically enriched environments to
enhance graph invariant learning for handling distribution shifts. In this
paper, we propose a novel approach to generate hierarchical semantic
environments for each graph. Firstly, given an input graph, we explicitly
extract variant subgraphs from the input graph to generate proxy predictions on
local environments. Then, stochastic attention mechanisms are employed to
re-extract the subgraphs for regenerating global environments in a hierarchical
manner. In addition, we introduce a new learning objective that guides our
model to learn the diversity of environments within the same hierarchy while
maintaining consistency across different hierarchies. This approach enables our
model to consider the relationships between environments and facilitates robust
graph invariant learning. Extensive experiments on real-world graph data have
demonstrated the effectiveness of our framework. Particularly, in the
challenging dataset DrugOOD, our method achieves up to 1.29\% and 2.83\%
improvement over the best baselines on IC50 and EC50 prediction tasks,
respectively.

本研究提出了一种新颖方法，通过生成层次化的语义环境来增强图不变性学习，以应对分布转移，实验证明该方法在真实世界的图数据上取得了显著效果。