We propose a method for high-performance semantic image segmentation (or
semantic pixel labelling) based on very deep residual networks, which achieves
the state-of-the-art performance. A few design factors are carefully considered
to this end.
We make the following contributions. (i) First, we evaluate different
variations of a fully convolutional residual network so as to find the best
configuration, including the number of layers, the resolution of feature maps,
and the size of field-of-view. Our experiments show that further enlarging the
field-of-view and increasing the resolution of feature maps are typically
beneficial, which however inevitably leads to a higher demand for GPU memories.
To walk around the limitation, we propose a new method to simulate a high
resolution network with a low resolution network, which can be applied during
training and/or testing. (ii) Second, we propose an online bootstrapping method
for training. We demonstrate that online bootstrapping is critically important
for achieving good accuracy. (iii) Third we apply the traditional dropout to
some of the residual blocks, which further improves the performance. (iv)
Finally, our method achieves the currently best mean intersection-over-union
78.3\% on the PASCAL VOC 2012 dataset, as well as on the recent dataset
Cityscapes.

本文提出了一种基于残差网络的高性能语义图像分割方法，并对不同变体的完全卷积残差网络进行了评估，进一步提出了用低分辨率网络模拟高分辨率网络的方法，引入了在线启动方法以及在一些残差块上应用的传统退化法，取得了在 PASCAL VOC 2012 数据集和 Cityscapes 上最好的平均重叠联合 75 的性能表现。

使用非常深的全卷积网络进行高性能语义分割

High-performance Semantic Segmentation Using Very Deep Fully  Convolutional Networks

Convolutional networks are powerful visual models that yield hierarchies of
features. We show that convolutional networks by themselves, trained
end-to-end, pixels-to-pixels, exceed the state-of-the-art in semantic
segmentation. Our key insight is to build "fully convolutional" networks that
take input of arbitrary size and produce correspondingly-sized output with
efficient inference and learning. We define and detail the space of fully
convolutional networks, explain their application to spatially dense prediction
tasks, and draw connections to prior models. We adapt contemporary
classification networks (AlexNet, the VGG net, and GoogLeNet) into fully
convolutional networks and transfer their learned representations by
fine-tuning to the segmentation task. We then define a novel architecture that
combines semantic information from a deep, coarse layer with appearance
information from a shallow, fine layer to produce accurate and detailed
segmentations. Our fully convolutional network achieves state-of-the-art
segmentation of PASCAL VOC (20% relative improvement to 62.2% mean IU on 2012),
NYUDv2, and SIFT Flow, while inference takes one third of a second for a
typical image.

该研究证明，卷积网络可以通过端到端的训练，从像素到像素地训练，超越先前的语义分割最先进方法，通过自适应大小输入和高效学习来生成相应大小的输出。该研究还详细介绍了全卷积网络的应用于空间密集预测任务，并将 AlexNet、VGG 网和 GoogLeNet 等现代分类网络改进成全卷积网络，并将它们学习到的表示通过微调应用于语义分割任务。最后，该研究提出了一种新的网络架构，将深层次粗糙层和浅层外观信息相结合以生成准确和详细的分割结果，在 PASCAL VOC、NYUDv2 和 SIFT Flow 数据集上实现了最先进的语义分割（2012 年的 mean IU 相较前一方法提高 20％），对典型图像的推理时间仅需三分之一秒。