We propose a novel method for semantic segmentation, the task of labeling each pixel in an image with a semantic class. Our method combines the advantages of the two main competing paradigms. Methods based on region classification offer proper spatial support for appearance measurements, but typically operate in two separate stages, none of which targets pixel labeling performance at the end of the pipeline. More recent fully convolutional methods are capable of end-to-end training for the final pixel labeling, but resort to fixed patches as spatial support. We show how to modify modern region-based approaches to enable end-to-end training for semantic segmentation. This is achieved via a differentiable region-to-pixel layer and a differentiable free-form Region-of-Interest pooling layer. Our method improves the state-of-the-art in terms of class-average accuracy with 64.0% on SIFT Flow and 49.9% on PASCAL Context, and is particularly accurate at object boundaries.

本文提出一种新的语义分割方法，该方法基于区域分类方法和全卷积方法相结合，能够实现端到端的培训。与目前现有的算法相比，该方法在语义分类的平均准确性方面提高了成绩，在SIFT Flow和PASCAL Context上达到了64.0%和49.9%的成绩，并且在物体边界方面特别准确。

基于区域的语义分割与端到端训练