Few-shot segmentation aims to train a segmentation model that can fast adapt
to novel classes with few exemplars. The conventional training paradigm is to
learn to make predictions on query images conditioned on the features from
support images. Previous methods only utilized the semantic-level prototypes of
support images as conditional information. These methods cannot utilize all
pixel-wise support information for the query predictions, which is however
critical for the segmentation task. In this paper, we focus on utilizing
pixel-wise relationships between support and query images to facilitate the
few-shot segmentation task. We design a novel Cycle-Consistent TRansformer
(CyCTR) module to aggregate pixel-wise support features into query ones. CyCTR
performs cross-attention between features from different images, i.e. support
and query images. We observe that there may exist unexpected irrelevant
pixel-level support features. Directly performing cross-attention may aggregate
these features from support to query and bias the query features. Thus, we
propose using a novel cycle-consistent attention mechanism to filter out
possible harmful support features and encourage query features to attend to the
most informative pixels from support images. Experiments on all few-shot
segmentation benchmarks demonstrate that our proposed CyCTR leads to remarkable
improvement compared to previous state-of-the-art methods. Specifically, on
Pascal-$5^i$ and COCO-$20^i$ datasets, we achieve 67.5% and 45.6% mIoU for
5-shot segmentation, outperforming previous state-of-the-art methods by 5.6%
and 7.1% respectively.

该研究提出了一种新的循环一致 Transformer（CyCTR）模块，通过交叉注意支持和查询图像的特征，聚合像素级支持特征到查询特征中，并使用循环一致的注意机制过滤可能有害的支持特征，最终在所有 few-shot 分割基准测试中实现了显着的改进。