Artificial neural networks thrive in solving the classification problem for a
particular rigid task, acquiring knowledge through generalized learning
behaviour from a distinct training phase. The resulting network resembles a
static entity of knowledge, with endeavours to extend this knowledge without
targeting the original task resulting in a catastrophic forgetting. Continual
learning shifts this paradigm towards networks that can continually accumulate
knowledge over different tasks without the need to retrain from scratch. We
focus on task incremental classification, where tasks arrive sequentially and
are delineated by clear boundaries. Our main contributions concern 1) a
taxonomy and extensive overview of the state-of-the-art, 2) a novel framework
to continually determine the stability-plasticity trade-off of the continual
learner, 3) a comprehensive experimental comparison of 11 state-of-the-art
continual learning methods and 4 baselines. We empirically scrutinize method
strengths and weaknesses on three benchmarks, considering Tiny Imagenet and
large-scale unbalanced iNaturalist and a sequence of recognition datasets. We
study the influence of model capacity, weight decay and dropout regularization,
and the order in which the tasks are presented, and qualitatively compare
methods in terms of required memory, computation time, and storage.

本文研究了基于任务增量分类的持续学习方法，提出了一种新的框架来确定持续学习者的稳定性与可塑性的权衡， 并通过三个基准测试以及综合实验比较了 11 种持续学习方法和 4 种基线的方法的强弱和影响。