Successful visual recognition networks benefit from aggregating information
spanning from a wide range of scales. Previous research has investigated
information fusion of connected layers or multiple branches in a block, seeking
to strengthen the power of multi-scale representations. Despite their great
successes, existing practices often allocate the neurons for each scale
manually, and keep the same ratio in all aggregation blocks of an entire
network, rendering suboptimal performance. In this paper, we propose to learn
the neuron allocation for aggregating multi-scale information in different
building blocks of a deep network. The most informative output neurons in each
block are preserved while others are discarded, and thus neurons for multiple
scales are competitively and adaptively allocated. Our scale aggregation
network (ScaleNet) is constructed by repeating a scale aggregation (SA) block
that concatenates feature maps at a wide range of scales. Feature maps for each
scale are generated by a stack of downsampling, convolution and upsampling
operations. The data-driven neuron allocation and SA block achieve strong
representational power at the cost of considerably low computational
complexity. The proposed ScaleNet, by replacing all 3x3 convolutions in ResNet
with our SA blocks, achieves better performance than ResNet and its outstanding
variants like ResNeXt and SE-ResNet, in the same computational complexity. On
ImageNet classification, ScaleNets absolutely reduce the top-1 error rate of
ResNets by 1.12 (101 layers) and 1.82 (50 layers). On COCO object detection,
ScaleNets absolutely improve the mmAP with backbone of ResNets by 3.6 (101
layers) and 4.6 (50 layers) on Faster RCNN, respectively. Code and models are
released at this https URL

本文提出了一种数据驱动的神经元分配方法，以自适应地在深度神经网络的不同构建块中汇总多尺度信息。通过使用基于堆栈的采样、卷积和上采样操作生成每个尺度的特征映射，探索神经元分配方法，对计算复杂度进行了大规模的实验测试，并成功地应用于图像分类和目标检测任务，在与 ResNet 和其变体的对比中， ScaleNet 能够获得更好的性能。