Semantic networks, such as the knowledge graph, can represent the knowledge
leveraging the graph structure. Although the knowledge graph shows promising
values in natural language processing, it suffers from incompleteness. This
paper focuses on knowledge graph completion by predicting linkage between
entities, which is a fundamental yet critical task. Semantic matching is a
potential solution as it can deal with unseen entities, which the translational
distance based methods struggle with. However, to achieve competitive
performance as translational distance based methods, semantic matching based
methods require large-scale datasets for the training purpose, which are
typically unavailable in practical settings. Therefore, we employ the language
model and introduce a novel knowledge graph architecture named LP-BERT, which
contains two main stages: multi-task pre-training and knowledge graph
fine-tuning. In the pre-training phase, three tasks are taken to drive the
model to learn the relationship from triples by predicting either entities or
relations. While in the fine-tuning phase, inspired by contrastive learning, we
design a triple-style negative sampling in a batch, which greatly increases the
proportion of negative sampling while keeping the training time almost
unchanged. Furthermore, we propose a new data augmentation method utilizing the
inverse relationship of triples to improve the performance and robustness of
the model. To demonstrate the effectiveness of our method, we conduct extensive
experiments on three widely-used datasets, WN18RR, FB15k-237, and UMLS. The
experimental results demonstrate the superiority of our methods, and our
approach achieves state-of-the-art results on WN18RR and FB15k-237 datasets.
Significantly, Hits@10 indicator is improved by 5% from previous
state-of-the-art result on the WN18RR dataset while reaching 100% on the UMLS
dataset.

本文研究如何提高知识图谱的完整性，通过引入 LP-BERT 模型，并结合语义匹配以及数据增强的方法，实现了知识图谱补全任务的最新成果，优于现有最佳表现。