Class activation maps (CAMs) are commonly employed in weakly supervised
semantic segmentation (WSSS) to produce pseudo-labels. Due to incomplete or
excessive class activation, existing studies often resort to offline CAM
refinement, introducing additional stages or proposing offline modules. This
can cause optimization difficulties for single-stage methods and limit
generalizability. In this study, we aim to reduce the observed CAM
inconsistency and error to mitigate reliance on refinement processes. We
propose an end-to-end WSSS model incorporating guided CAMs, wherein our
segmentation model is trained while concurrently optimizing CAMs online. Our
method, Co-training with Swapping Assignments (CoSA), leverages a dual-stream
framework, where one sub-network learns from the swapped assignments generated
by the other. We introduce three techniques: i) soft perplexity-based
regularization to penalize uncertain regions; ii) a threshold-searching
approach to dynamically revise the confidence threshold; and iii) contrastive
separation to address the coexistence problem. CoSA demonstrates exceptional
performance, achieving mIoU of 76.2\% and 51.0\% on VOC and COCO validation
datasets, respectively, surpassing existing baselines by a substantial margin.
Notably, CoSA is the first single-stage approach to outperform all existing
multi-stage methods including those with additional supervision. Code is
avilable at https://github.com/youshyee/CoSA.

提出了一种采用引导类激活图的端到端弱监督语义分割模型 CoSA，通过交替学习和三种技术的引入，有效减少了类激活图的不一致性和错误，达到了优于现有方法的性能。