We show that under mild conditions, Estimation of Distribution Algorithms (EDAs) can be written as variational Expectation-Maximization (EM) that uses a mixture of weighted particles as the approximate posterior. In the infinite particle limit, EDAs can be viewed as exact EM. Because EM sits on a rigorous statistical foundation and has been thoroughly analyzed, this connection provides a coherent framework with which to reason about EDAs. Importantly, the connection also suggests avenues for possible improvements to EDAs owing to our ability to leverage both general, and EM-specific statistical tools and generalizations. For example, we make use of results about known EM convergence properties to propose an adaptive, hybrid EDA-gradient descent algorithm; this hybrid demonstrates better performance than either component of the hybrid on several canonical, non-convex test functions. We also demonstrate empirically that although one might hypothesize that reducing the variational gap could prove useful, it actually degrades performance of EDAs. Finally, we show that the equivalence between EM and EDAs provides us with a new perspective on why EDAs are performing approximate natural gradient descent.

该论文提出了一种新的框架，将一类分布估计算法，特别是协方差矩阵适应算法，写成了蒙特卡罗期望最大化算法和无限样本极限下的精确 EM，这个发现为 EDAs 研究提供了一个基于 EM 坚实统计基础的新的、一致的框架。

基于期望最大化的分布估计算法视角