The performance of supervised semantic segmentation methods highly relies on
the availability of large-scale training data. To alleviate this dependence,
few-shot semantic segmentation (FSS) is introduced to leverage the model
trained on base classes with sufficient data into the segmentation of novel
classes with few data. FSS methods face the challenge of model generalization
on novel classes due to the distribution shift between base and novel classes.
To overcome this issue, we propose a class-shared memory (CSM) module
consisting of a set of learnable memory vectors. These memory vectors learn
elemental object patterns from base classes during training whilst re-encoding
query features during both training and inference, thereby improving the
distribution alignment between base and novel classes. Furthermore, to cope
with the performance degradation resulting from the intra-class variance across
images, we introduce an uncertainty-based feature augmentation (UFA) module to
produce diverse query features during training for improving the model's
robustness. We integrate CSM and UFA into representative FSS works, with
experimental results on the widely-used PASCAL-5$^i$ and COCO-20$^i$ datasets
demonstrating the superior performance of ours over state of the art.

我们提出了一种基于少样本语义分割的方法，利用具有充足数据的基础类别训练的模型进行对只有少量数据的新类别的分割，并引入了类别共享记忆模块和基于不确定性的特征增强模块来提高模型的泛化性能。实验结果表明，我们的方法在广泛使用的 PASCAL-5i 和 COCO-20i 数据集上相比最先进的方法具有卓越的性能。

基于记忆引导网络的不确定性特征增强方法在少样本语义分割中的应用

Memory-guided Network with Uncertainty-based Feature Augmentation for  Few-shot Semantic Segmentation

Modern supervised semantic segmentation methods are usually finetuned based
on the supervised or self-supervised models pre-trained on ImageNet. Recent
work shows that transferring the knowledge from CLIP to semantic segmentation
via prompt learning can achieve promising performance. The performance boost
comes from the feature enhancement with multimodal alignment, i.e., the dot
product between vision and text embeddings. However, how to improve the
multimodal alignment for better transfer performance in dense tasks remains
underexplored. In this work, we focus on improving the quality of vision-text
alignment from two aspects of prompting design and loss function, and present
an instance-conditioned prompting with contrastive learning (ICPC) framework.
First, compared with the static prompt designs, we reveal that dynamic
prompting conditioned on image content can more efficiently utilize the text
encoder for complex dense tasks. Second, we propose an align-guided contrastive
loss to refine the alignment of vision and text embeddings. We further propose
lightweight multi-scale alignment for better performance. Extensive experiments
on three large-scale datasets (ADE20K, COCO-Stuff10k, and ADE20K-Full)
demonstrate that ICPC brings consistent improvements across diverse backbones.
Taking ResNet-50 as an example, ICPC outperforms the state-of-the-art
counterpart by 1.71%, 1.05%, and 1.41% mIoU on the three datasets,
respectively.

通过改进提示设计和损失函数，我们提出了一种实例条件提示与对比学习（ICPC）框架，改善了视觉和文本嵌入对准的质量，并在三个大规模数据集上展示出改进的性能。

ICPC：基于对照学习的实例条件提示的语义分割

ICPC: Instance-Conditioned Prompting with Contrastive Learning for  Semantic Segmentation

In the past few years, in the context of fully-supervised semantic
segmentation, several losses -- such as cross-entropy and dice -- have emerged
as de facto standards to supervise neural networks. The Dice loss is an
interesting case, as it comes from the relaxation of the popular Dice
coefficient; one of the main evaluation metric in medical imaging applications.
In this paper, we first study theoretically the gradient of the dice loss,
showing that concretely it is a weighted negative of the ground truth, with a
very small dynamic range. This enables us, in the second part of this paper, to
mimic the supervision of the dice loss, through a simple element-wise
multiplication of the network output with a negative of the ground truth. This
rather surprising result sheds light on the practical supervision performed by
the dice loss during gradient descent. This can help the practitioner to
understand and interpret results while guiding researchers when designing new
losses.

研究神经网络的监督语义分割的标准之一 --Dice 损失函数的梯度，并表明它具有非常小的动态范围，进而通过简单的元素乘法模拟它的监督方法，这一结论有助于理解和解释结果同时指导新损失函数的设计。