Intelligent systems deployed in the real world suffer from catastrophic
forgetting when exposed to a sequence of tasks. Humans, on the other hand,
acquire, consolidate, and transfer knowledge between tasks that rarely
interfere with the consolidated knowledge. Accompanied by self-regulated
neurogenesis, continual learning in the brain is governed by a rich set of
neurophysiological processes that harbor different types of knowledge, which
are then integrated by conscious processing. Thus, inspired by the Global
Workspace Theory of conscious information access in the brain, we propose
TAMiL, a continual learning method that entails task-attention modules to
capture task-specific information from the common representation space. We
employ simple, undercomplete autoencoders to create a communication bottleneck
between the common representation space and the global workspace, allowing only
the task-relevant information to the global workspace, thus greatly reducing
task interference. Experimental results show that our method outperforms
state-of-the-art rehearsal-based and dynamic sparse approaches and bridges the
gap between fixed capacity and parameter isolation approaches while being
scalable. We also show that our method effectively mitigates catastrophic
forgetting while being well-calibrated with reduced task-recency bias.

本研究提出了 TAMiL 方法，应用任务注意模块和自动编码器捕捉共同表示空间中的任务特定信息，并在全局工作区中保留任务相关信息以缓解灾难性遗忘现象。实验结果表明，该方法优于现有基于重新玩彩和动态稀疏方法的连续学习方法。