Stochastic gradient descent (SGD) has been found to be surprisingly effective
in training a variety of deep neural networks. However, there is still a lack
of understanding on how and why SGD can train these complex networks towards a
global minimum. In this study, we establish the convergence of SGD to a global
minimum for nonconvex optimization problems that are commonly encountered in
neural network training. Our argument exploits the following two important
properties: 1) the training loss can achieve zero value (approximately), which
has been widely observed in deep learning; 2) SGD follows a star-convex path,
which is verified by various experiments in this paper. In such a context, our
analysis shows that SGD, although has long been considered as a randomized
algorithm, converges in an intrinsically deterministic manner to a global
minimum.

本研究证明了随机梯度下降法 (SGD) 可训练深度神经网络，甚至可以收敛于全局最小值。这一结果得益于多个实验验证了 SGD 可以遵循恒星凸轨迹和训练损失近似于零值等性质，并以新方式揭示了 SGD 以确定性方式收敛于全局最小值。