Learning from Text-Attributed Graphs (TAGs) has attracted significant
attention due to its wide range of real-world applications. The rapid evolution
of large language models (LLMs) has revolutionized the way we process textual
data, which indicates a strong potential to replace shallow text embedding
generally used in Graph Neural Networks (GNNs). However, we find that existing
LLM approaches that exploit text information in graphs suffer from inferior
computation and data efficiency. In this work, we introduce a novel and
efficient approach for the end-to-end fine-tuning of Large Language Models
(LLMs) on TAGs, named LEADING. The proposed approach maintains computation cost
and memory overhead comparable to the graph-less fine-tuning of LLMs. Moreover,
it transfers the rick knowledge in LLMs to downstream graph learning tasks
effectively with limited labeled data in semi-supervised learning. Its superior
computation and data efficiency are demonstrated through comprehensive
experiments, offering a promising solution for a wide range of LLMs and graph
learning tasks on TAGs.

通过提出一种新颖高效的方法 LEADING，实现了对大型语言模型在文本属性图（TAGs）上的端到端微调，具有比现有方法更高的计算和数据效率，为一系列大型语言模型和基于 TAGs 的图学习任务提供了有前景的解决方案。