Traditional knowledge graph embedding (KGE) methods typically require
preserving the entire knowledge graph (KG) with significant training costs when
new knowledge emerges. To address this issue, the continual knowledge graph
embedding (CKGE) task has been proposed to train the KGE model by learning
emerging knowledge efficiently while simultaneously preserving decent old
knowledge. However, the explicit graph structure in KGs, which is critical for
the above goal, has been heavily ignored by existing CKGE methods. On the one
hand, existing methods usually learn new triples in a random order, destroying
the inner structure of new KGs. On the other hand, old triples are preserved
with equal priority, failing to alleviate catastrophic forgetting effectively.
In this paper, we propose a competitive method for CKGE based on incremental
distillation (IncDE), which considers the full use of the explicit graph
structure in KGs. First, to optimize the learning order, we introduce a
hierarchical strategy, ranking new triples for layer-by-layer learning. By
employing the inter- and intra-hierarchical orders together, new triples are
grouped into layers based on the graph structure features. Secondly, to
preserve the old knowledge effectively, we devise a novel incremental
distillation mechanism, which facilitates the seamless transfer of entity
representations from the previous layer to the next one, promoting old
knowledge preservation. Finally, we adopt a two-stage training paradigm to
avoid the over-corruption of old knowledge influenced by under-trained new
knowledge. Experimental results demonstrate the superiority of IncDE over
state-of-the-art baselines. Notably, the incremental distillation mechanism
contributes to improvements of 0.2%-6.5% in the mean reciprocal rank (MRR)
score.

基于增量蒸馏的竞争性方法对 CKGE 进行了改进，通过充分利用知识图谱中的显式图结构，优化了学习顺序，并设计了一种新颖的增量蒸馏机制以有效保留旧知识，实验结果表明该方法在平均倒数排名得分上取得了 0.2%-6.5% 的改进。