Stochastic Gradient Langevin Dynamics (SGLD) is a popular variant of
Stochastic Gradient Descent, where properly scaled isotropic Gaussian noise is
added to an unbiased estimate of the gradient at each iteration. This modest
change allows SGLD to escape local minima and suffices to guarantee asymptotic
convergence to global minimizers for sufficiently regular non-convex objectives
(Gelfand and Mitter, 1991). The present work provides a nonasymptotic analysis
in the context of non-convex learning problems, giving finite-time guarantees
for SGLD to find approximate minimizers of both empirical and population risks.
As in the asymptotic setting, our analysis relates the discrete-time SGLD
Markov chain to a continuous-time diffusion process. A new tool that drives the
results is the use of weighted transportation cost inequalities to quantify the
rate of convergence of SGLD to a stationary distribution in the Euclidean
$2$-Wasserstein distance.

使用加权运输成本不等式来量化 SGLD 在欧几里得 2 - 瓦瑟斯坦距离下收敛到随机分布的速率，并在非凸学习问题的背景下提供有限时间保证来找到两种风险的近似最小化器。

随机梯度朗逊动力学的非凸学习：非渐近分析

Non-convex learning via Stochastic Gradient Langevin Dynamics: a  nonasymptotic analysis

In machine learning research, the proximal gradient methods are popular for
solving various optimization problems with non-smooth regularization. Inexact
proximal gradient methods are extremely important when exactly solving the
proximal operator is time-consuming, or the proximal operator does not have an
analytic solution. However, existing inexact proximal gradient methods only
consider convex problems. The knowledge of inexact proximal gradient methods in
the non-convex setting is very limited. % Moreover, for some machine learning
models, there is still no proposed solver for exactly solving the proximal
operator. To address this challenge, in this paper, we first propose three
inexact proximal gradient algorithms, including the basic version and
Nesterov's accelerated version. After that, we provide the theoretical analysis
to the basic and Nesterov's accelerated versions. The theoretical results show
that our inexact proximal gradient algorithms can have the same convergence
rates as the ones of exact proximal gradient algorithms in the non-convex
setting.
Finally, we show the applications of our inexact proximal gradient algorithms
on three representative non-convex learning problems. All experimental results
confirm the superiority of our new inexact proximal gradient algorithms.

本文研究了机器学习中基于不光滑正则化的各种优化问题，并提出了三种不精确的近端梯度算法，包括基本版本和 Nesterov 加速版本。理论分析表明，我们的不精确近端梯度算法可以在非凸情况下具有和精确近端梯度算法相同的收敛速度。实验结果证实了新算法在三个代表性非凸学习问题上的优越性。