This paper addresses the challenge of incremental learning in growing graphs
with increasingly complex tasks. The goal is to continually train a graph model
to handle new tasks while retaining its inference ability on previous tasks.
Existing methods usually neglect the importance of memory diversity, limiting
in effectively selecting high-quality memory from previous tasks and
remembering broad previous knowledge within the scarce memory on graphs. To
address that, we introduce a novel holistic Diversified Memory Selection and
Generation (DMSG) framework for incremental learning in graphs, which first
introduces a buffer selection strategy that considers both intra-class and
inter-class diversities, employing an efficient greedy algorithm for sampling
representative training nodes from graphs into memory buffers after learning
each new task. Then, to adequately rememorize the knowledge preserved in the
memory buffer when learning new tasks, we propose a diversified memory
generation replay method. This method first utilizes a variational layer to
generate the distribution of buffer node embeddings and sample synthesized ones
for replaying. Furthermore, an adversarial variational embedding learning
method and a reconstruction-based decoder are proposed to maintain the
integrity and consolidate the generalization of the synthesized node
embeddings, respectively. Finally, we evaluate our model on node classification
tasks involving increasing class numbers. Extensive experimental results on
publicly accessible datasets demonstrate the superiority of DMSG over
state-of-the-art methods.

本研究论文提出了一种应对日益复杂任务下增量学习的挑战的方法。通过引入全新的多样化内存选择和生成（DMSG）框架，该框架在图形学习中利用了缓冲选择和内存生成回放的策略来保证知识的一致性和概括性，进而有效地存储和利用长期记忆。实验证明了 DMSG 方法在节点分类任务中的出色性能。

增长图中的渐进学习的整体记忆多样化

Holistic Memory Diversification for Incremental Learning in Growing  Graphs

Recently, continual learning (CL) has gained significant interest because it
enables deep learning models to acquire new knowledge without forgetting
previously learnt information. However, most existing works require knowing the
task identities and boundaries, which is not realistic in a real context. In
this paper, we address a more challenging and realistic setting in CL, namely
the Task-Free Continual Learning (TFCL) in which a model is trained on
non-stationary data streams with no explicit task information. To address TFCL,
we introduce an evolved mixture model whose network architecture is dynamically
expanded to adapt to the data distribution shift. We implement this expansion
mechanism by evaluating the probability distance between the knowledge stored
in each mixture model component and the current memory buffer using the Hilbert
Schmidt Independence Criterion (HSIC). We further introduce two simple dropout
mechanisms to selectively remove stored examples in order to avoid memory
overload while preserving memory diversity. Empirical results demonstrate that
the proposed approach achieves excellent performance.

本文提出了一种解决深度学习模型在没有显式任务信息的非定常数据流上进行持续学习的方法，即基于进化的混合模型，该模型的网络架构动态扩展以适应数据分布的变化，并引入了两个简单的丢弃机制来避免存储的例子过多导致记忆过载问题，实证结果表明，该方法在持续学习任务上取得了优异的性能。