Exploiting multi-scale features has shown great potential in tackling
semantic segmentation problems. The aggregation is commonly done with sum or
concatenation (concat) followed by convolutional (conv) layers. However, it
fully passes down the high-level context to the following hierarchy without
considering their interrelation. In this work, we aim to enable the low-level
feature to aggregate the complementary context from adjacent high-level feature
maps by a cross-scale pixel-to-region relation operation. We leverage
cross-scale context propagation to make the long-range dependency capturable
even by the high-resolution low-level features. To this end, we employ an
efficient feature pyramid network to obtain multi-scale features. We propose a
Relational Semantics Extractor (RSE) and Relational Semantics Propagator (RSP)
for context extraction and propagation respectively. Then we stack several RSP
into an RSP head to achieve the progressive top-down distribution of the
context. Experiment results on two challenging datasets Cityscapes and COCO
demonstrate that the RSP head performs competitively on both semantic
segmentation and panoptic segmentation with high efficiency. It outperforms
DeeplabV3 [1] by 0.7% with 75% fewer FLOPs (multiply-adds) in the semantic
segmentation task.

本文提出了一种通过跨尺度像素到区域关系操作，使低层特征聚合相邻高层特征的互补上下文，发挥跨尺度上下文传播作用的有效特征塔网络方法，具有较高的效率和性能表现。