Stochastic variational inference with an amortized inference model and the reparameterization trick has become a widely-used algorithm for learning latent variable models. Increasing the flexibility of approximate posterior distributions while maintaining computational tractability is one of the core problems in stochastic variational inference. Two families of approaches proposed to address the problem are flow-based and multisample-based approaches such as importance weighted auto-encoders (IWAE). We introduce a new learning algorithm, the annealed importance weighted auto-encoder (AIWAE), for learning latent variable models. The proposed AIWAE combines multisample-based and flow-based approaches with the annealed importance sampling and its memory cost stays constant when the depth of flows increases. The flow constructed using an annealing process in AIWAE facilitates the exploration of the latent space when the posterior distribution has multiple modes. Through computational experiments, we show that, compared to models trained using the IWAE, AIWAE-trained models are better density models, have more complex posterior distributions and use more latent space representation capacity.

本文提出了使用退火重要性采样(annealed importance sampling)来学习深度生成模型的方法，该方法是变分推断(variational inference)和马尔可夫链蒙特卡罗(Markov chain Monte Carlo)两种主要的近似方法的结合，通过实验表明该方法比重要性加权自编码器(importance weighted auto-encoders)更好地建模了概率密度，并且通过权衡计算和模型精度的关系提高模型准确性的同时不增加内存成本。

使用退火重要性采样学习深度生成模型