Incremental learning aims to overcome catastrophic forgetting when learning
deep networks from sequential tasks. With impressive learning efficiency and
performance, prompt-based methods adopt a fixed backbone to sequential tasks by
learning task-specific prompts. However, existing prompt-based methods heavily
rely on strong pretraining (typically trained on ImageNet-21k), and we find
that their models could be trapped if the potential gap between the pretraining
task and unknown future tasks is large. In this work, we develop a learnable
Adaptive Prompt Generator (APG). The key is to unify the prompt retrieval and
prompt learning processes into a learnable prompt generator. Hence, the whole
prompting process can be optimized to reduce the negative effects of the gap
between tasks effectively. To make our APG avoid learning ineffective
knowledge, we maintain a knowledge pool to regularize APG with the feature
distribution of each class. Extensive experiments show that our method
significantly outperforms advanced methods in exemplar-free incremental
learning without (strong) pretraining. Besides, under strong retraining, our
method also has comparable performance to existing prompt-based models, showing
that our method can still benefit from pretraining. Codes can be found at
this https URL

本研究提出了一种可学习的自适应提示生成器（APG），通过将提示的检索和学习过程统一为可学习的提示生成器，以降低任务之间的差距对模型性能的负面影响，在无（强）预训练条件下，在递增学习中明显优于先进方法。