Deep Residual Neural Networks (ResNets) have demonstrated remarkable success across a wide range of real-world applications. In this paper, we identify a suitable scaling factor (denoted by $\alpha$) on the residual branch of deep wide ResNets to achieve good generalization ability. We show that if $\alpha$ is a constant, the class of functions induced by Residual Neural Tangent Kernel (RNTK) is asymptotically not learnable, as the depth goes to infinity. We also highlight a surprising phenomenon: even if we allow $\alpha$ to decrease with increasing depth $L$, the degeneration phenomenon may still occur. However, when $\alpha$ decreases rapidly with $L$, the kernel regression with deep RNTK with early stopping can achieve the minimax rate provided that the target regression function falls in the reproducing kernel Hilbert space associated with the infinite-depth RNTK. Our simulation studies on synthetic data and real classification tasks such as MNIST, CIFAR10 and CIFAR100 support our theoretical criteria for choosing $\alpha$.

深度残差神经网络（ResNets）在各种实际应用中取得了显著的成功。本文确定了适当的缩放因子（用$\alpha$表示），用于深度宽ResNets的残差分支，以实现良好的泛化能力。我们证明，如果$\alpha$是一个常数，则深度趋于无穷时，由残差神经切向核（RNTK）引起的函数类是渐近不可学习的。我们还强调了一个令人惊讶的现象：即使允许$\alpha$随着深度L的增加而减小，退化现象仍可能发生。然而，当$\alpha$与L迅速减小时，使用深度RNTK进行早停止的核回归可以达到最小最大速率，前提是目标回归函数位于与无穷深度RNTK相关联的再生核希尔伯特空间中。我们对合成数据和实际分类任务（如MNIST、CIFAR10和CIFAR100）的模拟研究支持我们对$\alpha$选择的理论标准。

利用合适的缩放因子提高深层宽残差网络的泛化能力