Stochastic approximation methods have recently gained popularity for variational inference, but many existing approaches treat them as "black-box" tools. Thus, they often do not take advantage of the geometry of the posterior and usually require a decreasing sequence of step-sizes (which converges slowly in practice). We introduce a new stochastic-approximation method that uses a proximal-gradient framework. Our method exploits the geometry and structure of the variational lower bound, and contains many existing methods, such as stochastic variational inference, as a special case. We establish the convergence of our method under a "non-decreasing" step-size schedule, which has both theoretical and practical advantages. We consider setting the step-size based on the continuity of the objective and the geometry of the posterior, and show that our method gives a faster rate of convergence for variational-Gaussian inference than existing stochastic methods.

提出了一种基于随机梯度方法的变分推断新方法，不仅利用变分参数空间的几何性质，而且即使对于非共轭模型也可以产生简单的闭合式更新，该方法也具有收敛速率分析，即使用于非凸目标的随机镜像下降。在多种问题上，实验证明新算法在该框架中导出可以导致最先进的结果。

使用带有一般散度函数的近端梯度方法加速随机变分推断