Existing knowledge graph (KG) embedding models have primarily focused on
static KGs. However, real-world KGs do not remain static, but rather evolve and
grow in tandem with the development of KG applications. Consequently, new facts
and previously unseen entities and relations continually emerge, necessitating
an embedding model that can quickly learn and transfer new knowledge through
growth. Motivated by this, we delve into an expanding field of KG embedding in
this paper, i.e., lifelong KG embedding. We consider knowledge transfer and
retention of the learning on growing snapshots of a KG without having to learn
embeddings from scratch. The proposed model includes a masked KG autoencoder
for embedding learning and update, with an embedding transfer strategy to
inject the learned knowledge into the new entity and relation embeddings, and
an embedding regularization method to avoid catastrophic forgetting. To
investigate the impacts of different aspects of KG growth, we construct four
datasets to evaluate the performance of lifelong KG embedding. Experimental
results show that the proposed model outperforms the state-of-the-art inductive
and lifelong embedding baselines.

本研究提出了一个新型知识图谱嵌入模型，实现快速学习和传输新的知识。通过使用掩码知识图自编码器进行嵌入学习和更新，并采用嵌入传输策略和嵌入规则化方法，避免灾难性遗忘。实验证明，该模型在生命周期嵌入与归纳嵌入方面表现优于现有模型。

不断嵌入学习和迁移，用于不断增长的知识图谱

Lifelong Embedding Learning and Transfer for Growing Knowledge Graphs

Embedding-based neural topic models could explicitly represent words and
topics by embedding them to a homogeneous feature space, which shows higher
interpretability. However, there are no explicit constraints for the training
of embeddings, leading to a larger optimization space. Also, a clear
description of the changes in embeddings and the impact on model performance is
still lacking. In this paper, we propose an embedding regularized neural topic
model, which applies the specially designed training constraints on word
embedding and topic embedding to reduce the optimization space of parameters.
To reveal the changes and roles of embeddings, we introduce \textbf{uniformity}
into the embedding-based neural topic model as the evaluation metric of
embedding space. On this basis, we describe how embeddings tend to change
during training via the changes in the uniformity of embeddings. Furthermore,
we demonstrate the impact of changes in embeddings in embedding-based neural
topic models through ablation studies. The results of experiments on two
mainstream datasets indicate that our model significantly outperforms baseline
models in terms of the harmony between topic quality and document modeling.
This work is the first attempt to exploit uniformity to explore changes in
embeddings of embedding-based neural topic models and their impact on model
performance to the best of our knowledge.

本研究提出了一种基于嵌入正则化约束的神经主题模型，引入了嵌入空间的均匀性度量，通过剔除不必要的参数优化空间，研究嵌入的变化对于模型性能的影响，以此实现主题质量和文档建模之间的平衡。