We provide the first convergence guarantee for full black-box variational
inference (BBVI), also known as Monte Carlo variational inference. While
preliminary investigations worked on simplified versions of BBVI (e.g., bounded
domain, bounded support, only optimizing for the scale, and such), our setup
does not need any such algorithmic modifications. Our results hold for
log-smooth posterior densities with and without strong log-concavity and the
location-scale variational family. Also, our analysis reveals that certain
algorithm design choices commonly employed in practice, particularly, nonlinear
parameterizations of the scale of the variational approximation, can result in
suboptimal convergence rates. Fortunately, running BBVI with proximal
stochastic gradient descent fixes these limitations, and thus achieves the
strongest known convergence rate guarantees. We evaluate this theoretical
insight by comparing proximal SGD against other standard implementations of
BBVI on large-scale Bayesian inference problems.

本文提供了第一篇关于全黑箱变分推断的收敛性保证，特别是蒙特卡罗变分推断。作者通过与传统算法相比的分析，证明了使用鲁棒的变分族文件和负责的算法设计，特别是使用近端随机梯度下降，可以实现最强的已知收敛速率保证。