Few-shot class-incremental learning (FSCIL) aims at learning to classify new
classes continually from limited samples without forgetting the old classes.
The mainstream framework tackling FSCIL is first to adopt the cross-entropy
(CE) loss for training at the base session, then freeze the feature extractor
to adapt to new classes. However, in this work, we find that the CE loss is not
ideal for the base session training as it suffers poor class separation in
terms of representations, which further degrades generalization to novel
classes. One tempting method to mitigate this problem is to apply an additional
naive supervised contrastive learning (SCL) in the base session. Unfortunately,
we find that although SCL can create a slightly better representation
separation among different base classes, it still struggles to separate base
classes and new classes. Inspired by the observations made, we propose
Semantic-Aware Virtual Contrastive model (SAVC), a novel method that
facilitates separation between new classes and base classes by introducing
virtual classes to SCL. These virtual classes, which are generated via
pre-defined transformations, not only act as placeholders for unseen classes in
the representation space, but also provide diverse semantic information. By
learning to recognize and contrast in the fantasy space fostered by virtual
classes, our SAVC significantly boosts base class separation and novel class
generalization, achieving new state-of-the-art performance on the three
widely-used FSCIL benchmark datasets. Code is available at:
this https URL

通过引入虚拟类别的概念，我们提出了一种基于语义感知的虚拟对比模型（SAVC），显著提高了基础类别的分类效果和新类别的泛化能力。

基于语义感知的虚拟对比约束学习：适用于小样本类递增学习的方法

Learning with Fantasy: Semantic-Aware Virtual Contrastive Constraint for  Few-Shot Class-Incremental Learning

This paper is on soft prompt learning for Vision \& Language (V&L) models.
Similarly to their NLP counterparts, V\&L models can be adapted to a downstream
task by learning soft continuous prompts using a few training examples. Current
methods learn the soft prompts by minimizing a cross-entropy loss using as
class weights the features obtained by passing the prompts plus the class names
through the text encoder. Such methods, however, significantly overfit the
training data suffering from large accuracy degradation when tested on unseen
classes from the same domain. Our main contribution, in this paper, is a
surprisingly simple approach to alleviate this problem: we use a second cross
entropy loss to minimize the distance between the learned soft prompts and a
set of hand-engineered manual prompts (obtained by prompt engineering). The
proposed loss can be interpreted in multiple ways including as a regularizer,
as a means for language-based augmentation, and as a way of learning more
discriminative class centroids. Importantly, our formulation is inherently
amenable to including, during training, virtual classes, i.e. class names for
which no visual samples are available, further increasing the robustness of the
learned prompts. Through extensive evaluations on 11 datasets, we show that our
approach (a) significantly outperforms all prior works on soft prompting, and
(b) matches and surpasses, for the first time, the accuracy on novel classes
obtained by hand-crafted prompts and CLIP for the majority of the test
datasets. Code will be made available.

本文介绍了一种软提示学习的方法，用于 Vision & Language 模型。该方法通过使用第二交叉熵损失来最小化学习的软提示与手工工程提示之间的距离，进而提高模型性能，并能够训练虚拟类。在 11 个数据集上进行的广泛评估表明，该方法显著优于所有先前的软提示工作，并在大多数测试数据集上匹配和超越手工制作提示和 CLIP 的新类准确性。

语言感知的视觉与语言基础模型软提示

Language-Aware Soft Prompting for Vision & Language Foundation Models

The core of deep metric learning (DML) involves learning visual similarities
in high-dimensional embedding space. One of the main challenges is to
generalize from seen classes of training data to unseen classes of test data.
Recent works have focused on exploiting past embeddings to increase the number
of instances for the seen classes. Such methods achieve performance improvement
via augmentation, while the strong focus on seen classes still remains. This
can be undesirable for DML, where training and test data exhibit entirely
different classes. In this work, we present a novel training strategy for DML
called MemVir. Unlike previous works, MemVir memorizes both embedding features
and class weights to utilize them as additional virtual classes. The
exploitation of virtual classes not only utilizes augmented information for
training but also alleviates a strong focus on seen classes for better
generalization. Moreover, we embed the idea of curriculum learning by slowly
adding virtual classes for a gradual increase in learning difficulty, which
improves the learning stability as well as the final performance. MemVir can be
easily applied to many existing loss functions without any modification.
Extensive experimental results on famous benchmarks demonstrate the superiority
of MemVir over state-of-the-art competitors. Code of MemVir is publicly
available.

本研究提出了一种新的深度度量学习的训练策略 MemVir，通过记忆嵌入特征和类别权重并利用它们作为额外的虚拟类别，实现了对虚拟类别的利用，从而提高了对未见类别的泛化能力，同时通过逐步增加虚拟类别来增加学习难度，改善了学习稳定性和最终性能。