The ability to learn new concepts continually is necessary in this ever-changing world. However, deep neural networks suffer from catastrophic forgetting when learning new categories. Many works have been proposed to alleviate this phenomenon, whereas most of them either fall into the stability-plasticity dilemma or take too much computation or storage overhead. Inspired by the gradient boosting algorithm to gradually fit the residuals between the target and the current approximation function, we propose a novel two-stage learning paradigm FOSTER, empowering the model to learn new categories adaptively. Specifically, we first dynamically expand new modules to fit the residuals of the target and the original model. Next, we remove redundant parameters and feature dimensions through an effective distillation strategy to maintain the single backbone model. We validate our method FOSTER on CIFAR-100, ImageNet-100/1000 under different settings. Experimental results show that our method achieves state-of-the-art performance.

提出了一种新的双阶段学习范式 FOSTER，通过动态模块扩展和有效的蒸馏策略来解决深度神经网络遗忘问题，实现自适应学习新类别，实验证明该方法在 CIFAR-100 和 ImageNet-100/1000 数据集下达到了最先进的性能。

FOSTER: 特征增强和压缩用于类增量学习