Few-shot semantic segmentation aims to segment the target objects in query
under the condition of a few annotated support images. Most previous works
strive to mine more effective category information from the support to match
with the corresponding objects in query. However, they all ignored the category
information gap between query and support images. If the objects in them show
large intra-class diversity, forcibly migrating the category information from
the support to the query is ineffective. To solve this problem, we are the
first to introduce an intermediate prototype for mining both deterministic
category information from the support and adaptive category knowledge from the
query. Specifically, we design an Intermediate Prototype Mining Transformer
(IPMT) to learn the prototype in an iterative way. In each IPMT layer, we
propagate the object information in both support and query features to the
prototype and then use it to activate the query feature map. By conducting this
process iteratively, both the intermediate prototype and the query feature can
be progressively improved. At last, the final query feature is used to yield
precise segmentation prediction. Extensive experiments on both PASCAL-5i and
COCO-20i datasets clearly verify the effectiveness of our IPMT and show that it
outperforms previous state-of-the-art methods by a large margin. Code is
available at this https URL

本研究提出了一种中介原型矿机器 (IPMT) 来学习和挖掘来自查询和支撑图像中的目标，通过迭代学习实现在查询和支撑图像间分类信息并结合中间原型进一步提高分割预测精确性。

用于小样本语义分割的中间原型挖掘变换器

Intermediate Prototype Mining Transformer for Few-Shot Semantic  Segmentation

Recently, some contrastive learning methods have been proposed to
simultaneously learn representations and clustering assignments, achieving
significant improvements. However, these methods do not take the category
information and clustering objective into consideration, thus the learned
representations are not optimal for clustering and the performance might be
limited. Towards this issue, we first propose a novel graph contrastive
learning framework, which is then applied to the clustering task and we come up
with the Graph Constrastive Clustering~(GCC) method. Different from basic
contrastive clustering that only assumes an image and its augmentation should
share similar representation and clustering assignments, we lift the
instance-level consistency to the cluster-level consistency with the assumption
that samples in one cluster and their augmentations should all be similar.
Specifically, on the one hand, the graph Laplacian based contrastive loss is
proposed to learn more discriminative and clustering-friendly features. On the
other hand, a novel graph-based contrastive learning strategy is proposed to
learn more compact clustering assignments. Both of them incorporate the latent
category information to reduce the intra-cluster variance while increasing the
inter-cluster variance. Experiments on six commonly used datasets demonstrate
the superiority of our proposed approach over the state-of-the-art methods.

本研究提出一种新的基于图的对比学习框架 ——Graph Contrastive Clustering（GCC），用于聚类任务，并利用该框架在六个常用数据集上进行了实验。实验结果表明，该方法在聚类问题上明显优于现有的最先进方法。

图形对比聚类

Graph Contrastive Clustering

We present novel method for image-text multi-modal representation learning.
In our knowledge, this work is the first approach of applying adversarial
learning concept to multi-modal learning and not exploiting image-text pair
information to learn multi-modal feature. We only use category information in
contrast with most previous methods using image-text pair information for
multi-modal embedding. In this paper, we show that multi-modal feature can be
achieved without image-text pair information and our method makes more similar
distribution with image and text in multi-modal feature space than other
methods which use image-text pair information. And we show our multi-modal
feature has universal semantic information, even though it was trained for
category prediction. Our model is end-to-end backpropagation, intuitive and
easily extended to other multi-modal learning work.

本文介绍了一种新的图像 - 文本多模态表示学习方法，该方法利用对抗学习概念进行多模态学习，不利用图像 - 文本对信息进行学习，只使用类别信息，其多模态特征的分布与利用图像 - 文本对信息进行学习的其他方法相比更为相似，并且具有通用的语义信息。该模型是端到端反向传播的，在其他多模态学习工作中易于扩展。