This paper studies the context aggregation problem in semantic image
segmentation. The existing researches focus on improving the pixel
representations by aggregating the contextual information within individual
images. Though impressive, these methods neglect the significance of the
representations of the pixels of the corresponding class beyond the input
image. To address this, this paper proposes to mine the contextual information
beyond individual images to further augment the pixel representations. We first
set up a feature memory module, which is updated dynamically during training,
to store the dataset-level representations of various categories. Then, we
learn class probability distribution of each pixel representation under the
supervision of the ground-truth segmentation. At last, the representation of
each pixel is augmented by aggregating the dataset-level representations based
on the corresponding class probability distribution. Furthermore, by utilizing
the stored dataset-level representations, we also propose a representation
consistent learning strategy to make the classification head better address
intra-class compactness and inter-class dispersion. The proposed method could
be effortlessly incorporated into existing segmentation frameworks (e.g., FCN,
PSPNet, OCRNet and DeepLabV3) and brings consistent performance improvements.
Mining contextual information beyond image allows us to report state-of-the-art
performance on various benchmarks: ADE20K, LIP, Cityscapes and COCO-Stuff.

本文研究语义图像分割中的上下文聚合问题，提出了一种基于特征存储模块和类概率分布学习的像素表示增强方法，利用跨图像的上下文信息进一步提高了性能。