Both performance and efficiency are important to semantic segmentation.
State-of-the-art semantic segmentation algorithms are mostly based on dilated
Fully Convolutional Networks (dilatedFCN), which adopt dilated convolutions in
the backbone networks to extract high-resolution feature maps for achieving
high-performance segmentation performance. However, due to many convolution
operations are conducted on the high-resolution feature maps, such
dilatedFCN-based methods result in large computational complexity and memory
consumption. To balance the performance and efficiency, there also exist
encoder-decoder structures that gradually recover the spatial information by
combining multi-level feature maps from the encoder. However, the performances
of existing encoder-decoder methods are far from comparable with the
dilatedFCN-based methods. In this paper, we propose the EfficientFCN, whose
backbone is a common ImageNet pre-trained network without any dilated
convolution. A holistically-guided decoder is introduced to obtain the
high-resolution semantic-rich feature maps via the multi-scale features from
the encoder. The decoding task is converted to novel codebook generation and
codeword assembly task, which takes advantages of the high-level and low-level
features from the encoder. Such a framework achieves comparable or even better
performance than state-of-the-art methods with only 1/3 of the computational
cost. Extensive experiments on PASCAL Context, PASCAL VOC, ADE20K validate the
effectiveness of the proposed EfficientFCN.

本文提出了一种新的语义分割方法: EfficientFCN, 相较于基于 dilatedFCN 的方法，该方法既可以取得相似的高精度分割效果，又能够在计算成本上做到只有原来的 1/3，这一成果达到了一种性能和效率的平衡。