This paper explores the properties of the plain Vision Transformer (ViT) for
Weakly-supervised Semantic Segmentation (WSSS). The class activation map (CAM)
is of critical importance for understanding a classification network and
launching WSSS. We observe that different attention heads of ViT focus on
different image areas. Thus a novel weight-based method is proposed to
end-to-end estimate the importance of attention heads, while the self-attention
maps are adaptively fused for high-quality CAM results that tend to have more
complete objects. Besides, we propose a ViT-based gradient clipping decoder for
online retraining with the CAM results to complete the WSSS task. We name this
plain Transformer-based Weakly-supervised learning framework WeakTr. It
achieves the state-of-the-art WSSS performance on standard benchmarks, i.e.,
78.4% mIoU on the val set of PASCAL VOC 2012 and 50.3% mIoU on the val set of
COCO 2014. Code is available at this https URL

本文探讨了 Vision Transformer (ViT) 在弱监督语义分割 (WSSS) 中的属性。在提出的 WeakTr 框架中，通过自适应融合自注意力图来具有更完整的对象的高质量 CAM 结果。在标准基准测试中，WeakTr 实现了最先进的 WSSS 性能，即在 PASCAL VOC 2012 的 val set 上达到 78.4% 的 mIoU，在 COCO 2014 的 val set 上达到 50.3% 的 mIoU。