A residual-networks family with hundreds or even thousands of layers
dominates major image recognition tasks, but building a network by simply
stacking residual blocks inevitably limits its optimization ability. This paper
proposes a novel residual-network architecture, Residual networks of Residual
networks (RoR), to dig the optimization ability of residual networks. RoR
substitutes optimizing residual mapping of residual mapping for optimizing
original residual mapping. In particular, RoR adds level-wise shortcut
connections upon original residual networks to promote the learning capability
of residual networks. More importantly, RoR can be applied to various kinds of
residual networks (ResNets, Pre-ResNets and WRN) and significantly boost their
performance. Our experiments demonstrate the effectiveness and versatility of
RoR, where it achieves the best performance in all residual-network-like
structures. Our RoR-3-WRN58-4+SD models achieve new state-of-the-art results on
CIFAR-10, CIFAR-100 and SVHN, with test errors 3.77%, 19.73% and 1.59%,
respectively. RoR-3 models also achieve state-of-the-art results compared to
ResNets on ImageNet data set.

本文提出一种新的残差网络架构 RoR，通过在现有的残差网络中添加层次化的快捷连接实现优化，从而提升其学习能力，进而提高性能，实验结果展示 RoR 在各种残差网络类型中的有效性和通用性，并在多个数据集上达到最佳性能表现。