Traditional knowledge graph embedding (KGE) methods typically require
preserving the entire knowledge graph (KG) with significant training costs when
new knowledge emerges. To address this issue, the continual knowledge graph
embedding (CKGE) task has been proposed to train the KGE model by learning
emerging knowledge efficiently while simultaneously preserving decent old
knowledge. However, the explicit graph structure in KGs, which is critical for
the above goal, has been heavily ignored by existing CKGE methods. On the one
hand, existing methods usually learn new triples in a random order, destroying
the inner structure of new KGs. On the other hand, old triples are preserved
with equal priority, failing to alleviate catastrophic forgetting effectively.
In this paper, we propose a competitive method for CKGE based on incremental
distillation (IncDE), which considers the full use of the explicit graph
structure in KGs. First, to optimize the learning order, we introduce a
hierarchical strategy, ranking new triples for layer-by-layer learning. By
employing the inter- and intra-hierarchical orders together, new triples are
grouped into layers based on the graph structure features. Secondly, to
preserve the old knowledge effectively, we devise a novel incremental
distillation mechanism, which facilitates the seamless transfer of entity
representations from the previous layer to the next one, promoting old
knowledge preservation. Finally, we adopt a two-stage training paradigm to
avoid the over-corruption of old knowledge influenced by under-trained new
knowledge. Experimental results demonstrate the superiority of IncDE over
state-of-the-art baselines. Notably, the incremental distillation mechanism
contributes to improvements of 0.2%-6.5% in the mean reciprocal rank (MRR)
score.

基于增量蒸馏的竞争性方法对 CKGE 进行了改进，通过充分利用知识图谱中的显式图结构，优化了学习顺序，并设计了一种新颖的增量蒸馏机制以有效保留旧知识，实验结果表明该方法在平均倒数排名得分上取得了 0.2%-6.5% 的改进。

基于递增蒸馏的持续知识图嵌入

Towards Continual Knowledge Graph Embedding via Incremental Distillation

In recent years, there has been a resurgence in methods that use distributed
(neural) representations to represent and reason about semantic knowledge for
robotics applications. However, while robots often observe previously unknown
concepts, these representations typically assume that all concepts are known a
priori, and incorporating new information requires all concepts to be learned
afresh. Our work relaxes this limiting assumption of existing representations
and tackles the incremental knowledge graph embedding problem by leveraging the
principles of a range of continual learning methods. Through an experimental
evaluation with several knowledge graphs and embedding representations, we
provide insights about trade-offs for practitioners to match a semantics-driven
robotics applications to a suitable continual knowledge graph embedding method.

本文介绍了一种基于连续学习方法的增量知识图嵌入技术，弥补了现有嵌入技术只能处理先前已知概念的局限性，并提供了一些在知识图嵌入技术和机器人应用之间进行权衡的洞见。