Continual learning (CL) aims to empower models to learn new tasks without
forgetting previously acquired knowledge. Most prior works concentrate on the
techniques of architectures, replay data, regularization, \etc. However, the
category name of each class is largely neglected. Existing methods commonly
utilize the one-hot labels and randomly initialize the classifier head. We
argue that the scarce semantic information conveyed by the one-hot labels
hampers the effective knowledge transfer across tasks. In this paper, we
revisit the role of the classifier head within the CL paradigm and replace the
classifier with semantic knowledge from pretrained language models (PLMs).
Specifically, we use PLMs to generate semantic targets for each class, which
are frozen and serve as supervision signals during training. Such targets fully
consider the semantic correlation between all classes across tasks. Empirical
studies show that our approach mitigates forgetting by alleviating
representation drifting and facilitating knowledge transfer across tasks. The
proposed method is simple to implement and can seamlessly be plugged into
existing methods with negligible adjustments. Extensive experiments based on
eleven mainstream baselines demonstrate the effectiveness and generalizability
of our approach to various protocols. For example, under the class-incremental
learning setting on ImageNet-100, our method significantly improves the Top-1
accuracy by 3.2\% to 6.1\% while reducing the forgetting rate by 2.6\% to
13.1\%.

利用预训练语言模型（PLMs）中的语义知识替换持续学习（CL）中的分类器，生成对每个类别的语义目标作为监督信号，以减少遗忘并促进知识传递。