During the last few years, significant attention has been paid to the
stochastic training of artificial neural networks, which is known as an
effective regularization approach that helps improve the generalization
capability of trained models. In this work, the method of modified equations is
applied to show that the residual network and its variants with noise injection
can be regarded as weak approximations of stochastic differential equations.
Such observations enable us to bridge the stochastic training processes with
the optimal control of backward Kolmogorov's equations. This not only offers a
novel perspective on the effects of regularization from the loss landscape
viewpoint but also sheds light on the design of more reliable and efficient
stochastic training strategies. As an example, we propose a new way to utilize
Bernoulli dropout within the plain residual network architecture and conduct
experiments on a real-world image classification task to substantiate our
theoretical findings.

本文研究了基于改进方程的方法，表明残差网络及其变体可以被视为弱逼近随机微分方程。从损失景观的角度提供了关于正则化效应的新视角，并为设计更可靠和高效的随机训练策略提供了启示。我们提出了一种利用伯努利丢弃来进行实验的新方法，从而验证了我们的理论发现。