Knowledge graphs (KG) are essential background knowledge providers in many
tasks. When designing models for KG-related tasks, one of the key tasks is to
devise the Knowledge Representation and Fusion (KRF) module that learns the
representation of elements from KGs and fuses them with task representations.
While due to the difference of KGs and perspectives to be considered during
fusion across tasks, duplicate and ad hoc KRF modules design are conducted
among tasks. In this paper, we propose a novel knowledge graph pretraining
model KGTransformer that could serve as a uniform KRF module in diverse
KG-related tasks. We pretrain KGTransformer with three self-supervised tasks
with sampled sub-graphs as input. For utilization, we propose a general
prompt-tuning mechanism regarding task data as a triple prompt to allow
flexible interactions between task KGs and task data. We evaluate pretrained
KGTransformer on three tasks, triple classification, zero-shot image
classification, and question answering. KGTransformer consistently achieves
better results than specifically designed task models. Through experiments, we
justify that the pretrained KGTransformer could be used off the shelf as a
general and effective KRF module across KG-related tasks. The code and datasets
are available at this https URL.

本文提出一种新型预训练知识图谱模型 KGTransformer 作为多种知识图谱相关任务的通用 KRF 模块，使用三个自监督任务对其进行预训练，并使用三项任务来评估其性能。实验结果表明，KGTransformer 优于特定任务模型并可作为一个通用和有效的 KRF 模块。