stochastic variational inference (SVI) lets us scale up bayesian computation
to massive data. It uses stochastic optimization to fit a variational
distribution, following easy-to-compute noisy natural gradients.
CheapSVRG is proposed as a new stochastic variance-reduction optimization scheme which achieves a linear convergence rate through a surrogate computation while also balancing computational complexity.