We propose a general purpose variational inference algorithm that forms a
natural counterpart of gradient descent for optimization. Our method
iteratively transports a set of particles to match the target distribution, by
applying a form of functional gradient descent that minimizes the KL
divergence. Empirical studies are performed on various real world models and
datasets, on which our method is competitive with existing state-of-the-art
methods. The derivation of our method is based on a new theoretical result that
connects the derivative of KL divergence under smooth transforms with Stein's
identity and a recently proposed kernelized Stein discrepancy, which is of
independent interest.

本文提出了用于优化的通用变分推理算法，它是梯度下降法的一种自然补充，可以通过一种函数梯度下降来最小化 KL 距离，从而迭代地传输一组粒子以匹配目标分布。经过在各种真实世界模型和数据集上的实证研究，我们的方法与现有的最先进的方法相竞争。我们方法的推导基于一个新的理论结果，它连接了平滑转换下 KL 距离的导数与 Stein's 恒等式以及最近提出的核化的 Stein 距离，这也具有独立的兴趣。

斯坦变分梯度下降：一种通用贝叶斯推断算法

Stein Variational Gradient Descent: A General Purpose Bayesian Inference  Algorithm

We derive a new discrepancy statistic for measuring differences between two
probability distributions based on combining Stein's identity with the
reproducing kernel Hilbert space theory. We apply our result to test how well a
probabilistic model fits a set of observations, and derive a new class of
powerful goodness-of-fit tests that are widely applicable for complex and high
dimensional distributions, even for those with computationally intractable
normalization constants. Both theoretical and empirical properties of our
methods are studied thoroughly.

本文提出了一种新的差异度量统计量，该统计量基于 Stein 的恒等式和再生核希尔伯特空间理论相结合。我们将其应用于测试概率模型与观测值的拟合程度，并派生了一类新的强大的适用于复杂和高维分布的拟合优度检验，即使对于具有计算难度的常数归一化分布亦如此。我们全面研究了方法的理论和实证特性。