Deep neural networks are known to be difficult to train due to the
instability of back-propagation. A deep \emph{residual network} (ResNet) with
identity loops remedies this by stabilizing gradient computations. We prove a
boosting theory for the ResNet architecture. We construct $T$ weak module
classifiers, each contains two of the $T$ layers, such that the combined strong
learner is a ResNet. Therefore, we introduce an alternative Deep ResNet
training algorithm, \emph{BoostResNet}, which is particularly suitable in
non-differentiable architectures. Our proposed algorithm merely requires a
sequential training of $T$ "shallow ResNets" which are inexpensive. We prove
that the training error decays exponentially with the depth $T$ if the
\emph{weak module classifiers} that we train perform slightly better than some
weak baseline. In other words, we propose a weak learning condition and prove a
boosting theory for ResNet under the weak learning condition. Our results apply
to general multi-class ResNets. A generalization error bound based on margin
theory is proved and suggests ResNet's resistant to overfitting under network
with $l_1$ norm bounded weights.

本文针对深度神经网络训练的不稳定性，提出具有强学习能力的 ResNet 架构，并在该条件下证明了 ResNet 的 Boosting 理论，同时提出 BoostResNet 训练算法来刻画 “浅层 ResNet” 的序列训练，推导出了 ResNet 的弱学习条件，并证明在深度为 $T$ 时，训练的误差随着 $T$ 指数下降。基于边缘理论证明了广义分类 ResNet 的泛化误差上限，并建议 ResNet 对边缘带 $l_1$ 约束的权重具有抵抗过拟合的能力。