Continual learning (CL) has remained a persistent challenge for deep neural
networks due to catastrophic forgetting (CF) of previously learned tasks.
Several techniques such as weight regularization, experience rehearsal, and
parameter isolation have been proposed to alleviate CF. Despite their relative
success, these research directions have predominantly remained orthogonal and
suffer from several shortcomings, while missing out on the advantages of
competing strategies. On the contrary, the brain continually learns,
accommodates, and transfers knowledge across tasks by simultaneously leveraging
several neurophysiological processes, including neurogenesis, active
forgetting, neuromodulation, metaplasticity, experience rehearsal, and
context-dependent gating, rarely resulting in CF. Inspired by how the brain
exploits multiple mechanisms concurrently, we propose TriRE, a novel CL
paradigm that encompasses retaining the most prominent neurons for each task,
revising and solidifying the extracted knowledge of current and past tasks, and
actively promoting less active neurons for subsequent tasks through rewinding
and relearning. Across CL settings, TriRE significantly reduces task
interference and surpasses different CL approaches considered in isolation.

借鉴大脑同时利用多种机制的方式，我们提出了 TriRE，一种新的持续学习范式，它包括保留每个任务最突出的神经元，修订和巩固当前和过去任务的提取知识，以及通过回溯和重新学习促进次要活跃神经元用于后续任务。在持续学习设置中，TriRE 显著减少了任务干扰，并超过了考虑独立评估的不同持续学习方法。