The human brain is capable of learning tasks sequentially mostly without
forgetting. However, deep neural networks (DNNs) suffer from catastrophic
forgetting when learning one task after another. We address this challenge
considering a class-incremental learning scenario where the DNN sees test data
without knowing the task from which this data originates. During training,
Continual-Prune-and-Select (CP&S) finds a subnetwork within the DNN that is
responsible for solving a given task. Then, during inference, CP&S selects the
correct subnetwork to make predictions for that task. A new task is learned by
training available neuronal connections of the DNN (previously untrained) to
create a new subnetwork by pruning, which can include previously trained
connections belonging to other subnetwork(s) because it does not update shared
connections. This enables to eliminate catastrophic forgetting by creating
specialized regions in the DNN that do not conflict with each other while still
allowing knowledge transfer across them. The CP&S strategy is implemented with
different subnetwork selection strategies, revealing superior performance to
state-of-the-art continual learning methods tested on various datasets
(CIFAR-100, CUB-200-2011, ImageNet-100 and ImageNet-1000). In particular, CP&S
is capable of sequentially learning 10 tasks from ImageNet-1000 keeping an
accuracy around 94% with negligible forgetting, a first-of-its-kind result in
class-incremental learning. To the best of the authors' knowledge, this
represents an improvement in accuracy above 10% when compared to the best
alternative method.

研究人类大脑能够通过 Continual-Prune-and-Select (CP&S) 的 subnetwork 选择策略，在不产生灾难性遗忘并能跨 subnetworks 传递知识的情况下，使 DNN 顺序学习十项任务且准确率在 94% 以上，是一项在类增量学习方面的优异结果。