This paper proposes two novel knowledge transfer techniques for
class-incremental learning (CIL). First, we propose data-free generative replay
(DF-GR) to mitigate catastrophic forgetting in CIL by using synthetic samples
from a generative model. In the conventional generative replay, the generative
model is pre-trained for old data and shared in extra memory for later
incremental learning. In our proposed DF-GR, we train a generative model from
scratch without using any training data, based on the pre-trained
classification model from the past, so we curtail the cost of sharing
pre-trained generative models. Second, we introduce dual-teacher information
distillation (DT-ID) for knowledge distillation from two teachers to one
student. In CIL, we use DT-ID to learn new classes incrementally based on the
pre-trained model for old classes and another model (pre-)trained on the new
data for new classes. We implemented the proposed schemes on top of one of the
state-of-the-art CIL methods and showed the performance improvement on
CIFAR-100 and ImageNet datasets.

本文提出两种用于类增量学习的新型知识传输技术：无数据生成式回放（DF-GR）和双教师信息蒸馏（DT-ID），并在 CIFAR-100 和 ImageNet 数据集上表现出性能提升。