Few-shot class-incremental learning (FSCIL) presents the primary challenge of
balancing underfitting to a new session's task and forgetting the tasks from
previous sessions. To address this challenge, we develop a simple yet powerful
learning scheme that integrates effective methods for each core component of
the FSCIL network, including the feature extractor, base session classifiers,
and incremental session classifiers. In feature extractor training, our goal is
to obtain balanced generic representations that benefit both current viewable
and unseen or past classes. To achieve this, we propose a balanced supervised
contrastive loss that effectively balances these two objectives. In terms of
classifiers, we analyze and emphasize the importance of unifying initialization
methods for both the base and incremental session classifiers. Our method
demonstrates outstanding ability for new task learning and preventing
forgetting on CUB200, CIFAR100, and miniImagenet datasets, with significant
improvements over previous state-of-the-art methods across diverse metrics. We
conduct experiments to analyze the significance and rationale behind our
approach and visualize the effectiveness of our representations on new tasks.
Furthermore, we conduct diverse ablation studies to analyze the effects of each
module.

本文提出了一种简单有效的学习方案，来解决 few-shot class-incremental learning 中平衡新任务的欠拟合和遗忘之前任务的挑战，并在 CUB200、CIFAR100 和 miniImagenet 数据集上展现了领先的性能。

针对小样本类增量学习的平衡监督对比学习

Balanced Supervised Contrastive Learning for Few-Shot Class-Incremental  Learning

Continual learning tackles the setting of learning different tasks
sequentially. Despite the lots of previous solutions, most of them still suffer
significant forgetting or expensive memory cost. In this work, targeted at
these problems, we first study the continual learning process through the lens
of information theory and observe that forgetting of a model stems from the
loss of \emph{information gain} on its parameters from the previous tasks when
learning a new task. From this viewpoint, we then propose a novel continual
learning approach called Bit-Level Information Preserving (BLIP) that preserves
the information gain on model parameters through updating the parameters at the
bit level, which can be conveniently implemented with parameter quantization.
More specifically, BLIP first trains a neural network with weight quantization
on the new incoming task and then estimates information gain on each parameter
provided by the task data to determine the bits to be frozen to prevent
forgetting. We conduct extensive experiments ranging from classification tasks
to reinforcement learning tasks, and the results show that our method produces
better or on par results comparing to previous state-of-the-arts. Indeed, BLIP
achieves close to zero forgetting while only requiring constant memory
overheads throughout continual learning.

本文针对连续学习中可能出现的遗忘和内存成本昂贵的问题，通过信息论的视角提出一种新的 Bit-Level Information Preserving (BLIP) 方法，通过参数量化更新模型参数，以保留先前任务对模型参数的信息增益来避免遗忘。经过大量实验，该方法在分类任务和强化学习任务中均表现出优异的效果，实现了几乎零遗忘，且仅需要恒定的内存开销。