The principle underlying most existing continual learning (CL) methods is to
prioritize stability by penalizing changes in parameters crucial to old tasks,
while allowing for plasticity in other parameters. The importance of weights
for each task can be determined either explicitly through learning a
task-specific mask during training (e.g., parameter isolation-based approaches)
or implicitly by introducing a regularization term (e.g., regularization-based
approaches). However, all these methods assume that the importance of weights
for each task is unknown prior to data exposure. In this paper, we propose
ScrollNet as a scrolling neural network for continual learning. ScrollNet can
be seen as a dynamic network that assigns the ranking of weight importance for
each task before data exposure, thus achieving a more favorable
stability-plasticity tradeoff during sequential task learning by reassigning
this ranking for different tasks. Additionally, we demonstrate that ScrollNet
can be combined with various CL methods, including regularization-based and
replay-based approaches. Experimental results on CIFAR100 and TinyImagenet
datasets show the effectiveness of our proposed method. We release our code at
this https URL

ScrollNet 是一种滚动神经网络，通过在数据曝光之前为每个任务分配权重重要性排名，从而实现序贯任务学习中更有利的稳定性和可塑性权衡。我们的实验结果表明 ScrollNet 与各种持续学习方法的结合方法是有效的。