The Stochastic Gradient Langevin Dynamics (SGLD) are popularly used to
approximate Bayesian posterior distributions in statistical learning procedures
with large-scale data. As opposed to many usual Markov chain Monte Carlo (MCMC)
algorithms, SGLD is not stationary with respect to the posterior distribution;
two sources of error appear: The first error is introduced by an
Euler--Maruyama discretisation of a Langevin diffusion process, the second
error comes from the data subsampling that enables its use in large-scale data
settings. In this work, we consider an idealised version of SGLD to analyse the
method's pure subsampling error that we then see as a best-case error for
diffusion-based subsampling MCMC methods. Indeed, we introduce and study the
Stochastic Gradient Langevin Diffusion (SGLDiff), a continuous-time Markov
process that follows the Langevin diffusion corresponding to a data subset and
switches this data subset after exponential waiting times. There, we show that
the Wasserstein distance between the posterior and the limiting distribution of
SGLDiff is bounded above by a fractional power of the mean waiting time.
Importantly, this fractional power does not depend on the dimension of the
state space. We bring our results into context with other analyses of SGLD.

在理想版本的 SGLD 之下，该文分析了 SGLD 的纯子采样误差，并将其视为扩散子采样 MCMC 方法的最佳情况下的误差。作者引入和研究了一种连续时间马尔可夫过程 SGLDiff，它在指数等待时间后随数据子集转移，并展示了后验分布与 SGLDiff 极限分布之间的 Wasserstein 距离上限。

随机梯度 Langevin 扩散中的子采样误差

Subsampling Error in Stochastic Gradient Langevin Diffusions

We analyze the posterior contraction rates of parameters in Bayesian models
via the Langevin diffusion process, in particular by controlling moments of the
stochastic process and taking limits. Analogous to the non-asymptotic analysis
of statistical M-estimators and stochastic optimization algorithms, our
contraction rates depend on the structure of the population log-likelihood
function, and stochastic perturbation bounds between the population and sample
log-likelihood functions. Convergence rates are determined by a non-linear
equation that relates the population-level structure to stochastic perturbation
terms, along with a term characterizing the diffusive behavior. Based on this
technique, we also prove non-asymptotic versions of a Bernstein-von-Mises
guarantee for the posterior. We illustrate this general theory by deriving
posterior convergence rates for various concrete examples, as well as
approximate posterior distributions computed using Langevin sampling
procedures.

通过控制随机过程的矩和取极限，我们通过 Langevin 扩散过程分析贝叶斯模型中参数的后验收缩速率，并特别关注了结构和随机摄动边界对于结果的影响，此外，基于该技术，我们也证明了后验的 Bernstein-von-Mises 保证的非渐近版本。

参数后验收缩速率的扩散过程视角

A Diffusion Process Perspective on Posterior Contraction Rates for  Parameters

Langevin diffusion processes and their discretizations are often used for
sampling from a target density. The most convenient framework for assessing the
quality of such a sampling scheme corresponds to smooth and strongly
log-concave densities defined on $\mathbb R^p$. The present work focuses on
this framework and studies the behavior of Monte Carlo algorithms based on
discretizations of the kinetic Langevin diffusion. We first prove the geometric
mixing property of the kinetic Langevin diffusion with a mixing rate that is,
in the overdamped regime, optimal in terms of its dependence on the condition
number. We then use this result for obtaining improved guarantees of sampling
using the kinetic Langevin Monte Carlo method, when the quality of sampling is
measured by the Wasserstein distance. We also consider the situation where the
Hessian of the log-density of the target distribution is Lipschitz-continuous.
In this case, we introduce a new discretization of the kinetic Langevin
diffusion and prove that this leads to a substantial improvement of the upper
bound on the sampling error measured in Wasserstein distance.

使用 Langevin 扩散过程进行离散化的蒙特卡洛算法可用于对光滑且强对数凹密度进行采样，本文主要研究了这个框架，并证明了基于 kinetic Langevin 扩散的 Monte Carlo 算法的混合性质和采样质量，进一步证明了 Hessian 矩阵 Lipschitz 连续的情况下，使用新的离散化方法可以显著提高采样误差的上界。