Deep neural networks suffer from catastrophic forgetting when continually
learning new concepts. In this paper, we analyze this problem from a data
imbalance point of view. We argue that the imbalance between old task and new
task data contributes to forgetting of the old tasks. Moreover, the increasing
imbalance ratio during incremental learning further aggravates the problem. To
address the dynamic imbalance issue, we propose Uniform Prototype Contrastive
Learning (UPCL), where uniform and compact features are learned. Specifically,
we generate a set of non-learnable uniform prototypes before each task starts.
Then we assign these uniform prototypes to each class and guide the feature
learning through prototype contrastive learning. We also dynamically adjust the
relative margin between old and new classes so that the feature distribution
will be maintained balanced and compact. Finally, we demonstrate through
extensive experiments that the proposed method achieves state-of-the-art
performance on several benchmark datasets including CIFAR100, ImageNet100 and
TinyImageNet.

通过数据不平衡的角度来分析深度神经网络在连续学习新概念时容易出现灾难性遗忘问题，提出了一种名为 Uniform Prototype Contrastive Learning (UPCL) 的方法，通过学习均匀且紧凑的特征来解决动态不平衡问题，并通过在多个基准数据集上的实验验证了其在性能上的优越性。