A common challenge in continual learning (CL) is catastrophic forgetting, where the performance on old tasks drops after new, additional tasks are learned. In this paper, we propose a novel framework called ReCL to slow down forgetting in CL. Our framework exploits an implicit bias of gradient-based neural networks due to which these converge to margin maximization points. Such convergence points allow us to reconstruct old data from previous tasks, which we then combine with the current training data. Our framework is flexible and can be applied on top of existing, state-of-the-art CL methods to slow down forgetting. We further demonstrate the performance gain from our framework across a large series of experiments, including different CL scenarios (class incremental, domain incremental, task incremental learning) different datasets (MNIST, CIFAR10), and different network architectures. Across all experiments, we find large performance gains through ReCL. To the best of our knowledge, our framework is the first to address catastrophic forgetting by leveraging models in CL as their own memory buffers.

本研究针对持续学习中的灾难性遗忘问题，提出了一种名为ReCL的新框架，旨在减缓遗忘现象。该框架利用梯度基础神经网络的隐含偏差，重建旧数据并与当前训练数据结合，从而提高模型的性能。实验表明，ReCL在多种持续学习场景和不同数据集上显著提升了性能，是首个通过利用模型自我作为记忆缓冲区来解决灾难性遗忘问题的方法。

减缓持续学习中的遗忘