In class-incremental learning (CIL) scenarios, the phenomenon of catastrophic
forgetting caused by the classifier's bias towards the current task has long
posed a significant challenge. It is mainly caused by the characteristic of
discriminative models. With the growing popularity of the generative
multi-modal models, we would explore replacing discriminative models with
generative ones for CIL. However, transitioning from discriminative to
generative models requires addressing two key challenges. The primary challenge
lies in transferring the generated textual information into the classification
of distinct categories. Additionally, it requires formulating the task of CIL
within a generative framework. To this end, we propose a novel generative
multi-modal model (GMM) framework for class-incremental learning. Our approach
directly generates labels for images using an adapted generative model. After
obtaining the detailed text, we use a text encoder to extract text features and
employ feature matching to determine the most similar label as the
classification prediction. In the conventional CIL settings, we achieve
significantly better results in long-sequence task scenarios. Under the
Few-shot CIL setting, we have improved by at least 14\% accuracy over all the
current state-of-the-art methods with significantly less forgetting. Our code
is available at https://github.com/DoubleClass/GMM.

我们提出了一种适用于类增量学习的新型生成多模态模型（GMM）框架，通过适应的生成模型直接生成图像的标签，利用文本编码器提取文本特征并使用特征匹配确定最相似的标签作为分类预测结果，在常规类增量学习任务和少样本类增量学习任务中，我们相比目前最先进的方法有了显着的改善。