Black box variational inference (BBVI) with reparameterization gradients
triggered the exploration of divergence measures other than the
Kullback-Leibler (KL) divergence, such as alpha divergences. In this paper, we
view BBVI with generalized divergences as a form of estimating the marginal
likelihood via biased importance sampling. The choice of divergence determines
a bias-variance trade-off between the tightness of a bound on the marginal
likelihood (low bias) and the variance of its gradient estimators. Drawing on
variational perturbation theory of statistical physics, we use these insights
to construct a family of new variational bounds. Enumerated by an odd integer
order $K$, this family captures the standard KL bound for $K=1$, and converges
to the exact marginal likelihood as $K\to\infty$. Compared to
alpha-divergences, our reparameterization gradients have a lower variance. We
show in experiments on Gaussian Processes and Variational Autoencoders that the
new bounds are more mass covering, and that the resulting posterior covariances
are closer to the true posterior and lead to higher likelihoods on held-out
data.

本文针对黑盒变分推断方法提出了基于广义差异度量的偏置重要性采样方法，并通过变分扰动理论构建了一族新的变分上界，对多个模型进行实验得出新的变分上界在数据拟合上更优