Stochastic natural gradient variational inference (NGVI) is a popular
posterior inference method with applications in various probabilistic models.
Despite its wide usage, little is known about the non-asymptotic convergence
rate in the \emph{stochastic} setting. We aim to lessen this gap and provide a
better understanding. For conjugate likelihoods, we prove the first
$\mathcal{O}(\frac{1}{T})$ non-asymptotic convergence rate of stochastic NGVI.
The complexity is no worse than stochastic gradient descent (\aka black-box
variational inference) and the rate likely has better constant dependency that
leads to faster convergence in practice. For non-conjugate likelihoods, we show
that stochastic NGVI with the canonical parameterization implicitly optimizes a
non-convex objective. Thus, a global convergence rate of
$\mathcal{O}(\frac{1}{T})$ is unlikely without some significant new
understanding of optimizing the ELBO using natural gradients.

随机自然梯度变分推断方法（NGVI）在各种概率模型中被广泛应用，本文对其非渐近收敛速度进行研究，证明了对于共轭似然情况下的非渐近收敛率为 1/T，而对于非共轭似然情况下的随机 NGVI，证明了它隐式地优化了一个非凸目标。