Continual learning (CL) algorithms strive to acquire new knowledge while preserving prior information. However, this stability-plasticity trade-off remains a central challenge. This paper introduces a framework that dissects this trade-off, offering valuable insights into CL algorithms. The Readout-Decomposition of Activation Change (RDAC) framework first addresses the stability-plasticity dilemma and its relation to catastrophic forgetting. It relates learning-induced activation changes in the range of prior readouts to the degree of stability and changes in the null space to the degree of plasticity. In deep non-linear networks tackling split-CIFAR-110 tasks, the framework clarifies the stability-plasticity trade-offs of the popular regularization algorithms Synaptic intelligence (SI), Elastic-weight consolidation (EWC), and learning without Forgetting (LwF), and replay-based algorithms Gradient episodic memory (GEM), and data replay. GEM and data replay preserved stability and plasticity, while SI, EWC, and LwF traded off plasticity for stability. The inability of the regularization algorithms to maintain plasticity was linked to them restricting the change of activations in the null space of the prior readout. Additionally, for one-hidden-layer linear neural networks, we derived a gradient decomposition algorithm to restrict activation change only in the range of the prior readouts, to maintain high stability while not further sacrificing plasticity. Results demonstrate that the algorithm maintained stability without significant plasticity loss. The RDAC framework informs the behavior of existing CL algorithms and paves the way for novel CL approaches. Finally, it sheds light on the connection between learning-induced activation/representation changes and the stability-plasticity dilemma, also offering insights into representational drift in biological systems.

连续学习算法中的稳定性和可塑性之间的权衡问题是本研究的核心挑战，本文引入了RDAC框架来解决这一问题，并研究了常用的正则化算法在稳定性和可塑性之间的权衡情况，在深层非线性网络和单隐藏层线性神经网络上进行了验证实验，结果表明该算法在保持稳定性的同时并没有明显损失可塑性，为现有的连续学习算法提供了有价值的见解，并为新的连续学习方法铺平了道路，在生物系统中对学习诱导的激活/表示变化与稳定性-可塑性困境的关系以及再现漂移问题也提供了新的视角。

在持续学习中平衡稳定性和可塑性：激活变化的输出分解（RDAC）框架