Self-supervised representation learning on text-attributed graphs, which aims
to create expressive and generalizable representations for various downstream
tasks, has received increasing research attention lately. However, existing
methods either struggle to capture the full extent of structural context
information or rely on task-specific training labels, which largely hampers
their effectiveness and generalizability in practice. To solve the problem of
self-supervised representation learning on text-attributed graphs, we develop a
novel Graph-Centric Language model -- GRENADE. Specifically, GRENADE exploits
the synergistic effect of both pre-trained language model and graph neural
network by optimizing with two specialized self-supervised learning algorithms:
graph-centric contrastive learning and graph-centric knowledge alignment. The
proposed graph-centric self-supervised learning algorithms effectively help
GRENADE to capture informative textual semantics as well as structural context
information on text-attributed graphs. Through extensive experiments, GRENADE
shows its superiority over state-of-the-art methods. Implementation is
available at https://github.com/bigheiniu/GRENADE.

通过图中心化的自监督学习算法，GRENADE 模型在文本属性图上能够有效地捕捉信息性的文本语义和结构上下文信息，出现了优于现有方法的结果。