We propose an adaptively weighted stochastic gradient Langevin dynamics algorithm (SGLD), so-called contour stochastic gradient Langevin dynamics (CSGLD), for Bayesian learning in big data statistics. The proposed algorithm is essentially a \emph{scalable dynamic importance sampler}, which automatically \emph{flattens} the target distribution such that the simulation for a multi-modal distribution can be greatly facilitated. Theoretically, we prove a stability condition and establish the asymptotic convergence of the self-adapting parameter to a {\it unique fixed-point}, regardless of the non-convexity of the original energy function; we also present an error analysis for the weighted averaging estimators. Empirically, the CSGLD algorithm is tested on multiple benchmark datasets including CIFAR10 and CIFAR100. The numerical results indicate its superiority over the existing state-of-the-art algorithms in training deep neural networks.

本论文提出了一种称为轮廓随机梯度Langevin动力学（CSGLD）的自适应加权随机梯度Langevin动力学算法，用于Bayesian学习在大数据的统计学中。该算法在多模态分布的模拟中具有很大的优势，并测试了CIFAR10和CIFAR100的性能。

多峰分布模拟的轮廓随机梯度 Langevin 动力学算法