Few-shot class incremental learning (FSCIL) portrays the problem of learning
new concepts gradually, where only a few examples per concept are available to
the learner. Due to the limited number of examples for training, the techniques
developed for standard incremental learning cannot be applied verbatim to
FSCIL. In this work, we introduce a distillation algorithm to address the
problem of FSCIL and propose to make use of semantic information during
training. To this end, we make use of word embeddings as semantic information
which is cheap to obtain and which facilitate the distillation process.
Furthermore, we propose a method based on an attention mechanism on multiple
parallel embeddings of visual data to align visual and semantic vectors, which
reduces issues related to catastrophic forgetting. Via experiments on
MiniImageNet, CUB200, and CIFAR100 dataset, we establish new state-of-the-art
results by outperforming existing approaches.

本研究提出一种用于解决 few-shot class incremental learning（FSCIL）问题的蒸馏算法，该算法利用语义信息进行训练，同时提出基于注意机制的方法来对齐视觉和语义向量，从而有效降低了灾难性遗忘的影响，并成功地在 MiniImageNet、CUB200 和 CIFAR100 数据集上创下了新的最优结果。

基于语义感知的少样本类别增量学习知识蒸馏

Semantic-aware Knowledge Distillation for Few-Shot Class-Incremental  Learning

This paper focuses on a new task, i.e., transplanting a
category-and-task-specific neural network to a generic, modular network without
strong supervision. We design a functionally interpretable structure for the
generic network. Like building LEGO blocks, we teach the generic network a new
category by directly transplanting the module corresponding to the category
from a pre-trained network with a few or even without sample annotations. Our
method incrementally adds new categories to the generic network but does not
affect representations of existing categories. In this way, our method breaks
the typical bottleneck of learning a net for massive tasks and categories,
i.e., the requirement of collecting samples for all tasks and categories at the
same time before the learning begins. Thus, we use a new distillation
algorithm, namely back-distillation, to overcome specific challenges of network
transplanting. Our method without training samples even outperformed the
baseline with 100 training samples.

本文提出了一种通过神经网络移植实现将特定任务和类别的神经网络移植到通用的模块化网络中的方法，并采用新的蒸馏算法 back-distillation 来克服网络移植的特定挑战。