Class incremental learning (CIL) aims to incrementally update a trained model
with the new classes of samples (plasticity) while retaining previously learned
ability (stability). To address the most challenging issue in this goal, i.e.,
catastrophic forgetting, the mainstream paradigm is memory-replay CIL, which
consolidates old knowledge by replaying a small number of old classes of
samples saved in the memory. Despite effectiveness, the inherent
destruction-reconstruction dynamics in memory-replay CIL are an intrinsic
limitation: if the old knowledge is severely destructed, it will be quite hard
to reconstruct the lossless counterpart. Our theoretical analysis shows that
the destruction of old knowledge can be effectively alleviated by balancing the
contribution of samples from the current phase and those saved in the memory.
Motivated by this theoretical finding, we propose a novel Balanced
Destruction-Reconstruction module (BDR) for memory-replay CIL, which can
achieve better knowledge reconstruction by reducing the degree of maximal
destruction of old knowledge. Specifically, to achieve a better balance between
old knowledge and new classes, the proposed BDR module takes into account two
factors: the variance in training status across different classes and the
quantity imbalance of samples from the current phase and memory. By dynamically
manipulating the gradient during training based on these factors, BDR can
effectively alleviate knowledge destruction and improve knowledge
reconstruction. Extensive experiments on a range of CIL benchmarks have shown
that as a lightweight plug-and-play module, BDR can significantly improve the
performance of existing state-of-the-art methods with good generalization.

通过平衡当前阶段的样本和内存中保存的样本的贡献，提出了一种新的平衡破坏与重建模块（BDR），用于记忆回放的类增量学习（CIL），通过减少旧知识的最大破坏程度，可以实现更好的知识重建。实验证明，作为一种轻量级即插即用模块，BDR 能显著提高现有最先进方法的性能，并具有良好的泛化能力。