We present variational inference with sequential sample-average approximation
(VISA), a method for approximate inference in computationally intensive models,
such as those based on numerical simulations. VISA extends importance-weighted
forward-KL variational inference by employing a sequence of sample-average
approximations, which are considered valid inside a trust region. This makes it
possible to reuse model evaluations across multiple gradient steps, thereby
reducing computational cost. We perform experiments on high-dimensional
Gaussians, Lotka-Volterra dynamics, and a Pickover attractor, which demonstrate
that VISA can achieve comparable approximation accuracy to standard
importance-weighted forward-KL variational inference with computational savings
of a factor two or more for conservatively chosen learning rates.

我们提出了基于顺序样本平均逼近（VISA）的变分推断方法，用于处理基于数值模拟的计算密集型模型。VISA 通过使用一系列样本平均逼近，在一个信任区域内计算合理，从而能够在多个梯度步骤中重复使用模型评估，从而降低计算成本。我们在高维高斯分布、Lotka-Volterra 动力学和 Pickover 吸引子上进行实验，结果表明 VISA 能够以至少两倍的计算节约达到与标准的重要性加权前向 KL 变分推断相当的逼近精度。