Recently, large language models (LLMs) have been widely researched in the
field of graph machine learning due to their outstanding abilities in language
comprehension and learning. However, the significant gap between natural
language tasks and topological structure modeling poses a nonnegligible
challenge. Specifically, since natural language descriptions are not sufficient
for LLMs to understand and process graph-structured data, fine-tuned LLMs
perform even worse than some traditional GNN models on graph tasks, lacking
inherent modeling capabilities for graph structures. Existing research overly
emphasizes LLMs' understanding of semantic information captured by external
models, while inadequately exploring graph topological structure modeling,
thereby overlooking the genuine capabilities that LLMs lack. Consequently, in
this paper, we introduce a new framework, LangTopo, which aligns graph
structure modeling with natural language understanding at the token level.
LangTopo quantifies the graph structure modeling capabilities of GNNs and LLMs
by constructing a codebook for the graph modality and performs consistency
maximization. This process aligns the text description of LLM with the
topological modeling of GNN, allowing LLM to learn the ability of GNN to
capture graph structures, enabling LLM to handle graph-structured data
independently. We demonstrate the effectiveness of our proposed method on
multiple datasets.

该论文介绍了一种新的框架 LangTopo，通过构建代码簿量化了 GNN 和 LLM 的图结构建模能力，并通过一致性最大化的过程将 LLM 的文本描述与 GNN 的拓扑建模对齐，使 LLM 能够学习捕捉图结构的能力，从而能够独立处理图结构化数据。通过在多个数据集上展示我们提出的方法的有效性。