Expectation Propagation (Minka, 2001) is a widely successful algorithm for
variational inference. EP is an iterative algorithm used to approximate
complicated distributions, typically to find a Gaussian approximation of
posterior distributions. In many applications of this type, EP performs
extremely well. Surprisingly, despite its widespread use, there are very few
theoretical guarantees on Gaussian EP, and it is quite poorly understood.
In order to analyze EP, we first introduce a variant of EP: averaged-EP
(aEP), which operates on a smaller parameter space. We then consider aEP and EP
in the limit of infinite data, where the overall contribution of each
likelihood term is small and where posteriors are almost Gaussian. In this
limit, we prove that the iterations of both aEP and EP are simple: they behave
like iterations of Newton's algorithm for finding the mode of a function. We
use this limit behavior to prove that EP is asymptotically exact, and to obtain
other insights into the dynamic behavior of EP: for example, that it may
diverge under poor initialization exactly like Newton's method. EP is a simple
algorithm to state, but a difficult one to study. Our results should facilitate
further research into the theoretical properties of this important method.

本研究针对 Expectation Propagation 和其依托的高斯近似方法进行研究，提出了一个操作参数空间更小的变体 averaged-EP，并在数据充分、后验近似为高斯分布的极限情况下证明了 EP 是渐近精确的，迭代行为类似于 Newton 算法，但也可能像 Newton 算法一样因初始误差导致发散。我们的研究进一步促进了对此重要算法理论性质的研究。