Stochastic Gradient Descent (SGD) is being used routinely for optimizing
non-convex functions. Yet, the standard convergence theory for SGD in the
smooth non-convex setting gives a slow sublinear convergence to a stationary
point. In this work, we provide several convergence theorems for SGD showing
convergence to a global minimum for non-convex problems satisfying some extra
structural assumptions. In particular, we focus on two large classes of
structured non-convex functions: (i) Quasar (Strongly) Convex functions (a
generalization of convex functions) and (ii) functions satisfying the
Polyak-Lojasiewicz condition (a generalization of strongly-convex functions).
Our analysis relies on an Expected Residual condition which we show is a
strictly weaker assumption than previously used growth conditions, expected
smoothness or bounded variance assumptions. We provide theoretical guarantees
for the convergence of SGD for different step-size selections including
constant, decreasing and the recently proposed stochastic Polyak step-size. In
addition, all of our analysis holds for the arbitrary sampling paradigm, and as
such, we give insights into the complexity of minibatching and determine an
optimal minibatch size. Finally, we show that for models that interpolate the
training data, we can dispense of our Expected Residual condition and give
state-of-the-art results in this setting.

本文研究了随机梯度下降（SGD）在优化非凸函数方面的应用，提出了一些收敛理论，说明了在满足结构性假设的非凸问题中，SGD 能够收敛到全局最小值，分析过程基于一个期望残差条件，相比之前的假设更加宽松。

结构化非凸函数的 SGD：学习率、小批量和插值

SGD for Structured Nonconvex Functions: Learning Rates, Minibatching and  Interpolation

Mean field type models describing the limiting behavior, as the number of
players tends to $+\infty$, of stochastic differential game problems, have been
recently introduced by J-M. Lasry and P-L. Lions. Numerical methods for the
approximation of the stationary and evolutive versions of such models have been
proposed by the authors in previous works . Convergence theorems for these
methods are proved under various assumptions

介绍了 J-M Lasry 和 P-L Lions 提出的描述随机微分博弈问题的极限行为的均场类型模型，作者在先前的作品中提出了这些模型的稳态和演化版本的近似方法，并在各种假设下证明了这些方法的收敛定理。