Current state-of-the-art methods for image segmentation form a dense image
representation where the color, shape and texture information are all processed
together inside a deep CNN. This however may not be ideal as they contain very
different type of information relevant for recognition. Here, we propose a new
two-stream CNN architecture for semantic segmentation that explicitly wires
shape information as a separate processing branch, i.e. shape stream, that
processes information in parallel to the classical stream. Key to this
architecture is a new type of gates that connect the intermediate layers of the
two streams. Specifically, we use the higher-level activations in the classical
stream to gate the lower-level activations in the shape stream, effectively
removing noise and helping the shape stream to only focus on processing the
relevant boundary-related information. This enables us to use a very shallow
architecture for the shape stream that operates on the image-level resolution.
Our experiments show that this leads to a highly effective architecture that
produces sharper predictions around object boundaries and significantly boosts
performance on thinner and smaller objects. Our method achieves
state-of-the-art performance on the Cityscapes benchmark, in terms of both mask
(mIoU) and boundary (F-score) quality, improving by 2% and 4% over strong
baselines.

本文提出了一种新的双流 CNN 架构，其中显式地将形状信息作为单独的处理分支，即形状流，以并行方式处理信息。此架构的关键是一种连接两个流的中间层的新型门，用于连接经典流的高级激活以控制形状流的低级激活，有效地消除噪声并帮助形状流仅关注处理相关边界信息。实验表明，该方法在 Cityscapes 基准上可以显着提高细小物体的性能，并取得了最先进的性能。