Recent advances on semantic image segmentation are achieved by training deep convolutional neural networks (DCNNs) on a large amount of labelled images. On the other hand, structured models such as conditional random fields (CRFs) are an effective tool for labelling tasks, which play an important role in that CRFs can capture the global context of the image as well as local neighbourhood information. Here we explore contextual information to improve semantic image segmentation by taking advantage of the strength of both DCNNs and CRFs. Specifically, we train CRFs whose potential functions are modelled by fully convolutional neural networks (FCNNs). The resulted deep conditional random fields (DCRFs) are thus able to learn complex feature representations; and during the course of learning, dependencies between the output variables are taken into account. As in conventional DCNNs, the training of our model is performed in an end-to-end fashion using back-propagation. Different from DCNNs, however, inference may be needed at each gradient descent iteration, which can be computationally very expensive since typically millions of iterations are required. To enable efficient training, we propose to use approximate training, namely, piecewise training of CRFs, avoiding repeated inference. We achieve very competitive results on the PASCAL VOC 2012 dataset for semantic segmentation. Our model is trained on the VOC dataset alone with no extra data; and we achieve an intersection-over-union score of 70.7 on its test set, which outperforms state-of-the-art results. Our superior results particularly highlight the usefulness of the learnt pairwise neighbourhood relations modelled by complex FCNNs.

本文提出了一种改进图像语义分割性能的方法，通过使用上下文信息学习Patch-Patch和Patch-Background的语义相关性，使用带有条件随机场的CNN对邻近patch之间的语义相关性进行建模，并采用高效的分段训练方法。同时，通过使用传统的多尺度图片输入和滑动金字塔池化等手段捕捉图像区域与背景的上下文信息，结果在多种流行的语义分割数据集上取得了最先进的性能，并在PASCAL VOC 2012数据集上取得了78.0的交叉联合分数。

深层结构模型语义分割的高效分段训练