In this paper, we provide a systematic study to the prevailing ResNet architecture by showing a connection from a general deeply-fused net view to ensembling. We start by empirically demonstrating the resemblance between the expanded form of the deeply-fused net and an ensemble of neural networks. Our empirical results uncover that the deepest network among the ensemble components does not contribute the most significantly to the overall performance and instead it provides a manner to introduce many layers and thus guarantee the ensemble size. Guided by the above study and observation, we develop a new deeply-fused network that combines two networks in a \emph{merge-and-run} fusion manner. It is less deep than a ResNet with the same number of parameters but yields an ensemble of the same number of more-capable component networks, thus improving the classification accuracy. We evaluate the proposed network on the standard recognition tasks. Our approach demonstrates consistent improvements over the ResNet with the comparable setup, and achieves the state-of-the-art results (e.g., $3.57\%$ testing error on CIFAR-$10$, $19.00\%$ on CIFAR-$100$, $1.51\%$ on SVHN).

提出了一种简单的深度 merge-and-run 神经网络架构，通过一种线性幂等函数的组装方式，在并行的残差分支中进行输入平均和残差输入并行处理，不断添加到后续残差块的输出上，大大缩短路径，并提高信息流动和网络的训练难度。与 ResNet 相比，本方法在标准识别任务上表现出更好的性能。

带有合并运行映射的深度卷积神经网络