The emergence of large-scale pre-trained language models, such as ChatGPT,
has revolutionized various research fields in artificial intelligence.
Transformers-based large language models (LLMs) have gradually replaced CNNs
and RNNs to unify fields of computer vision and natural language processing.
Compared with the data that exists relatively independently such as images,
videos or texts, graph is a type of data that contains rich structural and
relational information. Meanwhile, natural language, as one of the most
expressive mediums, excels in describing complex structures. However, existing
work on incorporating graph learning problems into the generative language
modeling framework remains very limited. As the importance of language models
continues to grow, it becomes essential to explore whether LLMs can also
replace GNNs as the foundational model for graphs. In this paper, we propose
InstructGLM (Instruction-finetuned Graph Language Model), systematically design
highly scalable prompts based on natural language instructions, and use natural
language to describe the geometric structure and node features of the graph for
instruction tuning an LLMs to perform learning and inference on graphs in a
generative manner. Our method exceeds all competitive GNN baselines on
ogbn-arxiv, Cora and PubMed datasets, which demonstrates the effectiveness of
our method and sheds light on generative language models replacing GNNs as the
foundation model for graph machine learning.

我们提出了 InstructGLM（指导调整的图语言模型），基于自然语言说明系统地设计高度可扩展的提示，使用自然语言描述图的几何结构和节点特征，通过指导调优语言模型以一种生成方式在图上执行学习和推理，超过了所有竞争的 GNN 基准数据集，证明了我们方法的有效性，并为将生成语言模型替代 GNN 作为图机器学习的基础模型提供了启示。