Exemplar-Free Class Incremental Learning (EFCIL) aims to learn from a
sequence of tasks without having access to previous task data. In this paper,
we consider the challenging Cold Start scenario in which insufficient data is
available in the first task to learn a high-quality backbone. This is
especially challenging for EFCIL since it requires high plasticity, which
results in feature drift which is difficult to compensate for in the
exemplar-free setting. To address this problem, we propose a simple and
effective approach that consolidates feature representations by regularizing
drift in directions highly relevant to previous tasks and employs prototypes to
reduce task-recency bias. Our method, called Elastic Feature Consolidation
(EFC), exploits a tractable second-order approximation of feature drift based
on an Empirical Feature Matrix (EFM). The EFM induces a pseudo-metric in
feature space which we use to regularize feature drift in important directions
and to update Gaussian prototypes used in a novel asymmetric cross entropy loss
which effectively balances prototype rehearsal with data from new tasks.
Experimental results on CIFAR-100, Tiny-ImageNet, ImageNet-Subset and
ImageNet-1K demonstrate that Elastic Feature Consolidation is better able to
learn new tasks by maintaining model plasticity and significantly outperform
the state-of-the-art.

Elastic Feature Consolidation (EFC) 通过对于 feature drift 的正则化，以及使用高度相关的方向和高斯原型来更新重要方向的特征漂移，成功解决了 Exemplar-Free Class Incremental Learning (EFCIL) 在 Cold Start 情景下的挑战，并在学习新任务时保持了模型的可塑性，明显优于最先进方法。

冷启动免示例增量学习的弹性特征整合

Elastic Feature Consolidation for Cold Start Exemplar-free Incremental  Learning

In this work, we improve the generative replay in a continual learning
setting to perform well on challenging scenarios. Current generative rehearsal
methods are usually benchmarked on small and simple datasets as they are not
powerful enough to generate more complex data with a greater number of classes.
We notice that in VAE-based generative replay, this could be attributed to the
fact that the generated features are far from the original ones when mapped to
the latent space. Therefore, we propose three modifications that allow the
model to learn and generate complex data. More specifically, we incorporate the
distillation in latent space between the current and previous models to reduce
feature drift. Additionally, a latent matching for the reconstruction and
original data is proposed to improve generated features alignment. Further,
based on the observation that the reconstructions are better for preserving
knowledge, we add the cycling of generations through the previously trained
model to make them closer to the original data. Our method outperforms other
generative replay methods in various scenarios. Code available at
this https URL

改进了生成回放在连续学习环境中的表现，提出了三种修改方法以克服当前方法对于复杂数据生成的不足，并通过实验证明了方法在各种场景下的优越性能。

回顾过去：持续学习中生成回放的知识更好保留

Looking through the past: better knowledge retention for generative  replay in continual learning

Most recent graph clustering methods have resorted to Graph Auto-Encoders
(GAEs) to perform joint clustering and embedding learning. However, two
critical issues have been overlooked. First, the accumulative error, inflicted
by learning with noisy clustering assignments, degrades the effectiveness and
robustness of the clustering model. This problem is called Feature Randomness.
Second, reconstructing the adjacency matrix sets the model to learn irrelevant
similarities for the clustering task. This problem is called Feature Drift.
Interestingly, the theoretical relation between the aforementioned problems has
not yet been investigated. We study these issues from two aspects: (1) there is
a trade-off between Feature Randomness and Feature Drift when clustering and
reconstruction are performed at the same level, and (2) the problem of Feature
Drift is more pronounced for GAE models, compared with vanilla auto-encoder
models, due to the graph convolutional operation and the graph decoding design.
Motivated by these findings, we reformulate the GAE-based clustering
methodology. Our solution is two-fold. First, we propose a sampling operator
$\Xi$ that triggers a protection mechanism against the noisy clustering
assignments. Second, we propose an operator $\Upsilon$ that triggers a
correction mechanism against Feature Drift by gradually transforming the
reconstructed graph into a clustering-oriented one. As principal advantages,
our solution grants a considerable improvement in clustering effectiveness and
robustness and can be easily tailored to existing GAE models.

为了解决图聚类中的关键问题：特性随机性和特性漂移，本文提出了一种 GAE 聚类方法，并提出采样和操作符，以提高聚类效果和鲁棒性。