It has been observed that neural networks perform poorly when the data or
tasks are presented sequentially. Unlike humans, neural networks suffer greatly
from catastrophic forgetting, making it impossible to perform life-long
learning. To address this issue, memory-based continual learning has been
actively studied and stands out as one of the best-performing methods. We
examine memory-based continual learning and identify that large variation in
the representation space is crucial for avoiding catastrophic forgetting.
Motivated by this, we propose to diversify representations by using two types
of perturbations: model-agnostic variation (i.e., the variation is generated
without the knowledge of the learned neural network) and model-based variation
(i.e., the variation is conditioned on the learned neural network). We
demonstrate that enlarging representational variation serves as a general
principle to improve continual learning. Finally, we perform empirical studies
which demonstrate that our method, as a simple plug-and-play component, can
consistently improve a number of memory-based continual learning methods by a
large margin.

本篇研究论文介绍了记忆基础连续学习的特点和不足，并提出加强表示差异化的两种扰动方法，以改进连续学习的性能，最终基于大量实证研究结果成功验证了我们的结论。