It has been widely known that CAM (Class Activation Map) usually only
activates discriminative object regions and falsely includes lots of
object-related backgrounds. As only a fixed set of image-level object labels
are available to the WSSS (weakly supervised semantic segmentation) model, it
could be very difficult to suppress those diverse background regions consisting
of open set objects. In this paper, we propose a novel Cross Language Image
Matching (CLIMS) framework, based on the recently introduced Contrastive
Language-Image Pre-training (CLIP) model, for WSSS. The core idea of our
framework is to introduce natural language supervision to activate more
complete object regions and suppress closely-related open background regions.
In particular, we design object, background region and text label matching
losses to guide the model to excite more reasonable object regions for CAM of
each category. In addition, we design a co-occurring background suppression
loss to prevent the model from activating closely-related background regions,
with a predefined set of class-related background text descriptions. These
designs enable the proposed CLIMS to generate a more complete and compact
activation map for the target objects. Extensive experiments on PASCAL VOC2012
dataset show that our CLIMS significantly outperforms the previous
state-of-the-art methods.

本文提出了一种基于 CLIP 模型的跨语言图像匹配框架 CLIMS，通过自然语言监督以激活更合理的物体区域和抑制相关的背景区域，并使用一组预定义的类相关背景文本描述来防止模型激活相关背景区域，在 PASCAL VOC2012 数据集上进行了充分的实验，该方法明显优于之前的最先进方法。

弱监督语义分割的跨语言图像匹配

Cross Language Image Matching for Weakly Supervised Semantic Segmentation

Deep learning architectures exhibit a critical drop of performance due to
catastrophic forgetting when they are required to incrementally learn new
tasks. Contemporary incremental learning frameworks focus on image
classification and object detection while in this work we formally introduce
the incremental learning problem for semantic segmentation in which a
pixel-wise labeling is considered. To tackle this task we propose to distill
the knowledge of the previous model to retain the information about previously
learned classes, whilst updating the current model to learn the new ones. We
propose various approaches working both on the output logits and on
intermediate features. In opposition to some recent frameworks, we do not store
any image from previously learned classes and only the last model is needed to
preserve high accuracy on these classes. The experimental evaluation on the
Pascal VOC2012 dataset shows the effectiveness of the proposed approaches.

本研究旨在解决深度学习架构在面临新增任务时由于灾难性遗忘导致性能下降的问题，提出了一种知识蒸馏方法来保留以前学习过的类别信息，并同时学习新类别，本文在 Pascal VOC2012 数据集上进行了实验验证其有效性。