We propose a simple algorithm to train stochastic neural networks to draw
samples from given target distributions for probabilistic inference. Our method
is based on iteratively adjusting the neural network parameters so that the
output changes along a Stein variational gradient direction (Liu & Wang, 2016)
that maximally decreases the KL divergence with the target distribution. Our
method works for any target distribution specified by their unnormalized
density function, and can train any black-box architectures that are
differentiable in terms of the parameters we want to adapt. We demonstrate our
method with a number of applications, including variational autoencoder (VAE)
with expressive encoders to model complex latent space structures, and
hyper-parameter learning of MCMC samplers that allows Bayesian inference to
adaptively improve itself when seeing more data.

我们提出了一种简单的算法，用于训练随机神经网络从给定的目标分布中绘制样本，以进行概率推断。我们的方法基于按照 Stein 变分梯度方向（Liu & Wang，2016）调整神经网络参数，从而最大程度地减少与目标分布的 KL 散度。这种方法适用于任何由其未归一化密度函数指定的目标分布，并且可以训练任何可微分的参数结构。我们用许多应用程序演示了我们的方法，包括具有表现力编码器的可变分自编码器（VAE）以建模复杂的潜在空间结构，以及 MCMC 采样器的超参数学习，这允许贝叶斯推断在看到更多数据时自适应地改善。

用摊余式 Stein 变分梯度下降学习绘制样本

Learning to Draw Samples with Amortized Stein Variational Gradient  Descent

We propose a simple algorithm to train stochastic neural networks to draw
samples from given target distributions for probabilistic inference. Our method
is based on iteratively adjusting the neural network parameters so that the
output changes along a Stein variational gradient that maximumly decreases the
KL divergence with the target distribution. Our method works for any target
distribution specified by their unnormalized density function, and can train
any black-box architectures that are differentiable in terms of the parameters
we want to adapt. As an application of our method, we propose an amortized MLE
algorithm for training deep energy model, where a neural sampler is adaptively
trained to approximate the likelihood function. Our method mimics an
adversarial game between the deep energy model and the neural sampler, and
obtains realistic-looking images competitive with the state-of-the-art results.

本论文提出了一种利用随机神经网络训练概率推理目标分布的简单算法，该方法基于 Stein 变分梯度迭代地适应神经网络参数，并将输出沿着与目标分布 KL 散度最大程度下降的 Stein 变分梯度变化，可适用于任何由未归一化密度函数指定的目标分布，并且可以训练任何针对我们想要适应的参数可微不可微黑盒结构。作为我们方法的应用，我们提出了一种用于训练深度能量模型的分摊极大似然估计算法，其中自适应地训练神经采样器来近似似然函数。我们的方法模仿深度能量模型和神经采样器之间的对抗游戏，并获得与最先进结果竞争的逼真图像。