Feature pyramid networks (FPN) are widely exploited for multi-scale feature
fusion in existing advanced object detection frameworks. Numerous previous
works have developed various structures for bidirectional feature fusion, all
of which are shown to improve the detection performance effectively. We observe
that these complicated network structures require feature pyramids to be
stacked in a fixed order, which introduces longer pipelines and reduces the
inference speed. Moreover, semantics from non-adjacent levels are diluted in
the feature pyramid since only features at adjacent pyramid levels are merged
by the local fusion operation in a sequence manner. To address these issues, we
propose a novel architecture named RCNet, which consists of Reverse Feature
Pyramid (RevFP) and Cross-scale Shift Network (CSN). RevFP utilizes local
bidirectional feature fusion to simplify the bidirectional pyramid inference
pipeline. CSN directly propagates representations to both adjacent and
non-adjacent levels to enable multi-scale features more correlative. Extensive
experiments on the MS COCO dataset demonstrate RCNet can consistently bring
significant improvements over both one-stage and two-stage detectors with
subtle extra computational overhead. In particular, RetinaNet is boosted to
40.2 AP, which is 3.7 points higher than baseline, by replacing FPN with our
proposed model. On COCO test-dev, RCNet can achieve very competitive
performance with a single-model single-scale 50.5 AP. Codes will be made
available.

本篇研究提出一种新型的 RCNet 架构，由 Reverse Feature Pyramid (RevFP) 和 Cross-scale Shift Network (CSN) 组成，用于多尺度特征融合，在 MS COCO 测试集中证明 RCNet 能够显著提高 RetinaNet 等检测器的精度