Amortized variational inference (AVI) replaces instance-specific local inference with a global inference network. While AVI has enabled efficient training of deep generative models such as variational autoencoders (VAE), recent empirical work suggests that inference networks can produce suboptimal variational parameters. We propose a hybrid approach, to use AVI to initialize the variational parameters and run stochastic variational inference (SVI) to refine them. Crucially, the local SVI procedure is itself differentiable, so the inference network and generative model can be trained end-to-end with gradient-based optimization. This semi-amortized approach enables the use of rich generative models without experiencing the posterior-collapse phenomenon common in training VAEs for problems like text generation. Experiments show this approach outperforms strong autoregressive and variational baselines on standard text and image datasets.

我们提出了一种半摊分解方法，使用摊分变分推断(Amortized Variational Inference，AVI)来初始化变分参数并运行随机变分推断(Stochastic Variational Inference，SVI)来细化它们，从而使推理网络和生成模型能够进行端到端的梯度优化训练，这种方法有助于在像文本生成这样的问题上使用丰富的生成模型并避免训练VAE时产生的后验崩溃现象，实验证明这种方法在标准的文本和图像数据集上胜过强有力的自回归和变分基线。

半摊分变分自编码器