Convolutional networks are powerful visual models that yield hierarchies of
features. We show that convolutional networks by themselves, trained
end-to-end, pixels-to-pixels, exceed the state-of-the-art in semantic
segmentation. Our key insight is to build "fully convolutional" networks that
take input of arbitrary size and produce correspondingly-sized output with
efficient inference and learning. We define and detail the space of fully
convolutional networks, explain their application to spatially dense prediction
tasks, and draw connections to prior models. We adapt contemporary
classification networks (AlexNet, the VGG net, and GoogLeNet) into fully
convolutional networks and transfer their learned representations by
fine-tuning to the segmentation task. We then define a novel architecture that
combines semantic information from a deep, coarse layer with appearance
information from a shallow, fine layer to produce accurate and detailed
segmentations. Our fully convolutional network achieves state-of-the-art
segmentation of PASCAL VOC (20% relative improvement to 62.2% mean IU on 2012),
NYUDv2, and SIFT Flow, while inference takes one third of a second for a
typical image.

该研究证明，卷积网络可以通过端到端的训练，从像素到像素地训练，超越先前的语义分割最先进方法，通过自适应大小输入和高效学习来生成相应大小的输出。该研究还详细介绍了全卷积网络的应用于空间密集预测任务，并将 AlexNet、VGG 网和 GoogLeNet 等现代分类网络改进成全卷积网络，并将它们学习到的表示通过微调应用于语义分割任务。最后，该研究提出了一种新的网络架构，将深层次粗糙层和浅层外观信息相结合以生成准确和详细的分割结果，在 PASCAL VOC、NYUDv2 和 SIFT Flow 数据集上实现了最先进的语义分割（2012 年的 mean IU 相较前一方法提高 20％），对典型图像的推理时间仅需三分之一秒。