In continual learning, there is a serious problem of catastrophic forgetting, in which previous knowledge is forgotten when a model learns new tasks. Various methods have been proposed to solve this problem. Replay methods which replay data from previous tasks in later training, have shown good accuracy. However, replay methods have a generalizability problem from a limited memory buffer. In this paper, we tried to solve this problem by acquiring transferable knowledge through self-distillation using highly generalizable output in shallow layer as a teacher. Furthermore, when we deal with a large number of classes or challenging data, there is a risk of learning not converging and not experiencing overfitting. Therefore, we attempted to achieve more efficient and thorough learning by prioritizing the storage of easily misclassified samples through a new method of memory update. We confirmed that our proposed method outperformed conventional methods by experiments on CIFAR10, CIFAR100, and MiniimageNet datasets.

本研究针对连续学习中严重的灾难性遗忘问题，即模型在学习新任务时会遗忘以前的知识。论文提出通过自蒸馏获取可迁移知识，并优化记忆更新方法以优先存储易混淆样本，从而实现更高效的学习。实验结果表明，该方法在CIFAR10、CIFAR100和MiniimageNet数据集上的表现优于传统方法。

通过自蒸馏减少在线类别增量学习中的灾难性遗忘