In this note, we demonstrate a first-of-its-kind provable convergence of SGD
to the global minima of appropriately regularized logistic empirical risk of
depth $2$ nets -- for arbitrary data and with any number of gates with
adequately smooth and bounded activations like sigmoid and tanh. We also prove
an exponentially fast convergence rate for continuous time SGD that also
applies to smooth unbounded activations like SoftPlus. Our key idea is to show
the existence of Frobenius norm regularized logistic loss functions on
constant-sized neural nets which are "Villani functions" and thus be able to
build on recent progress with analyzing SGD on such objectives.

通过分析随机梯度下降算法在 Frobenius 范数正则化的 $logistic$ 损失函数上的最新进展，本文首次证明了对于任意数据和具有适当平滑和有界激活函数（如 $sigmoid$ 和 $tanh$）的门数量的 $2$ 层神经网络，SGD 能收敛到适当正则化的全局最小值；同时证明了连续时间 SGD 的指数快速收敛性，也适用于平滑的无界激活函数（如 $SoftPlus$）。