Foundation models like ChatGPT and GPT-4 have revolutionized artificial intelligence, exhibiting remarkable abilities to generalize across a wide array of tasks and applications beyond their initial training objectives. However, when this concept is applied to graph learning, a stark contrast emerges. Graph learning has predominantly focused on single-graph models, tailored to specific tasks or datasets, lacking the ability to transfer learned knowledge to different domains. This limitation stems from the inherent complexity and diversity of graph structures, along with the different feature and label spaces specific to graph data. In this paper, we present our UniGraph framework, designed to train a graph foundation model capable of generalizing to unseen graphs and tasks across diverse domains. Unlike single-graph models that use pre-computed node features of varying dimensions as input, our approach leverages Text-Attributed Graphs (TAGs) for unifying node representations. We propose a cascaded architecture of Language Models (LMs) and Graph Neural Networks (GNNs) as backbone networks with a self-supervised training objective based on Masked Graph Modeling (MGM). We introduce graph instruction tuning using Large Language Models (LLMs) to enable zero-shot prediction ability. Our comprehensive experiments across various graph learning tasks and domains demonstrate the model's effectiveness in self-supervised representation learning on unseen graphs, few-shot in-context transfer, and zero-shot transfer, even surpassing or matching the performance of GNNs that have undergone supervised training on target datasets.

我们提出了UniGraph框架，通过使用文本属性图（TAGs）来训练图基础模型，实现对未见过的图和任务在不同领域中的泛化能力。我们的方法采用了级联的语言模型（LMs）和图神经网络（GNNs）构建的骨干网络以及基于掩码图建模（MGM）的自我监督训练目标，并引入大型语言模型（LLMs）进行图指令调优，进而使模型能够进行零样本预测。在各种图学习任务和领域的全面实验证明了我们模型在未见过的图的自我监督表示学习、少样本上下文转移和零样本转移方面的有效性，甚至超越或匹配了在目标数据集上进行受监督训练的GNNs的性能。

UniGraph：从自然语言中学习跨领域图基础模型