In this paper, we present the ``joint pre-training and local re-training''
framework for learning and applying multi-source knowledge graph (KG)
embeddings. We are motivated by the fact that different KGs contain
complementary information to improve KG embeddings and downstream tasks. We
pre-train a large teacher KG embedding model over linked multi-source KGs and
distill knowledge to train a student model for a task-specific KG. To enable
knowledge transfer across different KGs, we use entity alignment to build a
linked subgraph for connecting the pre-trained KGs and the target KG. The
linked subgraph is re-trained for three-level knowledge distillation from the
teacher to the student, i.e., feature knowledge distillation, network knowledge
distillation, and prediction knowledge distillation, to generate more
expressive embeddings. The teacher model can be reused for different target KGs
and tasks without having to train from scratch. We conduct extensive
experiments to demonstrate the effectiveness and efficiency of our framework.

本文提出了 “联合预训练和本地重新训练” 框架，用于学习和应用多源知识图谱嵌入。旨在利用不同的知识图谱，改善知识图谱嵌入和下游任务。通过实体对齐构建链接的子图，通过三级知识蒸馏进行重新训练，实现知识的传递，生成更具表现力的嵌入。该框架在广泛实验中证明了其有效性和效率。