Performing inference and learning of deep generative networks in a Bayesian setting is desirable, where a sparsity-inducing prior can be adopted on model parameters or a nonparametric Bayesian process can be used to infer the network structure. However, posterior inference for such deep models is an extremely challenging task, which has largely not been well-addressed. In this paper, we present doubly stochastic gradient-based MCMC, a simple and effective method that can be widely applied for Bayesian inference of deep generative models in continuous parameter spaces. The algorithm is doubly stochastic in the sense that at each MCMC sampling step a mini-batch of data samples are randomly drawn to estimate the gradient of log-posterior and the intractable expectation over latent variables is further estimated via a Monte Carlo sampler. We demonstrate the effectiveness on learning deep sigmoid belief networks (DSBNs). Compared to the state-of-the-art methods using Gibbs sampling with data augmentation, our algorithm is much more efficient and manages to learn DSBNs on large datasets.

本文介绍了倍增随机梯度MCMC这一简单通用的方法，用于在折叠的连续参数空间中对深度生成模型进行（近似）贝叶斯推理。我们的方法不仅适用于密度估计和数据生成的任务，还可以用于缺失数据的填充，且在性能方面优于许多现有的竞争对手。

使用双重随机MCMC学习深度生成模型