Learning transferable representation of knowledge graphs (KGs) is challenging
due to the heterogeneous, multi-relational nature of graph structures. Inspired
by Transformer-based pretrained language models' success on learning
transferable representation for texts, we introduce a novel inductive KG
representation model (iHT) for KG completion by large-scale pre-training. iHT
consists of a entity encoder (e.g., BERT) and a neighbor-aware relational
scoring function both parameterized by Transformers. We first pre-train iHT on
a large KG dataset, Wikidata5M. Our approach achieves new state-of-the-art
results on matched evaluations, with a relative improvement of more than 25% in
mean reciprocal rank over previous SOTA models. When further fine-tuned on
smaller KGs with either entity and relational shifts, pre-trained iHT
representations are shown to be transferable, significantly improving the
performance on FB15K-237 and WN18RR.

文章介绍了一种名为 iHT 的知识图谱 (KG) 表征模型，该模型基于 Transformer 并经过大规模预训练，由实体编码器和邻居感知关系得分函数组成，可用于 KG 补全，经过实验证明其具有很好的泛化能力和良好的表现。