It is well known that Markov chain Monte Carlo (MCMC) methods scale poorly with dataset size. We compare the performance of two classes of methods which aim to solve this issue: stochastic gradient MCMC (SGMCMC), and divide and conquer methods. We find an SGMCMC method, stochastic gradient Langevin dynamics (SGLD) to be the most robust in these comparisons. This method makes use of a noisy estimate of the gradient of the log posterior, which significantly reduces the per iteration computational cost of the algorithm. We analyse the algorithm over different dataset sizes and show, despite the per iteration saving, the computational cost is still proportional to the dataset size. We use control variates, a method to reduce the variance in Monte Carlo estimates, to reduce this computational cost to $O(1)$. Next we show that a different control variate technique, known as zero variance control variates can be applied to SGMCMC algorithms for free. This post-processing step improves the inference of the algorithm by reducing the variance of the MCMC output. Zero variance control variates rely on the gradient of the log posterior; we explore how the variance reduction is affected by replacing this with the noisy gradient estimate calculated by SGMCMC.

本文研究马尔可夫链蒙特卡罗、随机梯度 Langevin 动力学、控制变量、计算成本和零方差控制变量等相关问题，提出了一种用于减少方差的 log 后验梯度估计方法，并分析了其在 SGLD 中的应用，发现在目标分布满足对数凹的假设下，对于任意数据集规模的情况下，其运算成本与数据集大小无关，并且研究了零方差控制变量技术的应用，可以通过减少 MCMC 输出的方差来提高算法的推断效果，并分析了由 SGMCMC 计算的嘈杂的梯度估计量对方差减少的影响。

随机梯度 MCMC 的控制变量