Multi-Class Incremental Learning (MCIL) aims to learn new concepts by
incrementally updating a model trained on previous concepts. However, there is
an inherent trade-off to effectively learning new concepts without catastrophic
forgetting of previous ones. To alleviate this issue, it has been proposed to
keep around a few examples of the previous concepts but the effectiveness of
this approach heavily depends on the representativeness of these examples. This
paper proposes a novel and automatic framework we call mnemonics, where we
parameterize exemplars and make them optimizable in an end-to-end manner. We
train the framework through bilevel optimizations, i.e., model-level and
exemplar-level. We conduct extensive experiments on three MCIL benchmarks,
CIFAR-100, ImageNet-Subset and ImageNet, and show that using mnemonics
exemplars can surpass the state-of-the-art by a large margin. Interestingly and
quite intriguingly, the mnemonics exemplars tend to be on the boundaries
between different classes.

本文提出了一种名为 “mnemonics” 自动化框架，通过维护先前类别的几个代表性示例，通过双层优化方式对其进行训练，探讨了较好的多类增量学习策略。mnemonics 在 CIFAR-100，ImageNet-Subset 和 ImageNet 三个 MCIL 基准上的实验结果表明，mnemonics 示例可大幅领先于现有技术。同时，mnemonics 的实例往往在不同类别之间的边界上，具有极大的研究兴趣。