In real-world applications, dynamic scenarios require the models to possess
the capability to learn new tasks continuously without forgetting the old
knowledge. Experience-Replay methods store a subset of the old images for joint
training. In the scenario of more strict privacy protection, storing the old
images becomes infeasible, which leads to a more severe plasticity-stability
dilemma and classifier bias. To meet the above challenges, we propose a new
architecture, named continual expansion and absorption transformer~(CEAT). The
model can learn the novel knowledge by extending the expanded-fusion layers in
parallel with the frozen previous parameters. After the task ends, we
losslessly absorb the extended parameters into the backbone to ensure that the
number of parameters remains constant. To improve the learning ability of the
model, we designed a novel prototype contrastive loss to reduce the overlap
between old and new classes in the feature space. Besides, to address the
classifier bias towards the new classes, we propose a novel approach to
generate the pseudo-features to correct the classifier. We experiment with our
methods on three standard Non-Exemplar Class-Incremental Learning~(NECIL)
benchmarks. Extensive experiments demonstrate that our model gets a significant
improvement compared with the previous works and achieves 5.38%, 5.20%, and
4.92% improvement on CIFAR-100, TinyImageNet, and ImageNet-Subset.

通过延展融合层并与冻结先前参数并行，然后无损地将扩展参数吸收到主干网络中，以确保参数数量恒定，同时采用新型的原型对比损失减少特征空间中新旧类别的重叠，并提出一种生成伪特征以校正分类器的新方法。实验结果表明，在 CIFAR-100、TinyImageNet 和 ImageNet-Subset 上，与之前的研究相比，我们的模型获得了显著的改进，分别提高了 5.38％、5.20％和 4.92％。