Mixture models serve as one fundamental tool with versatile applications.
However, their training techniques, like the popular Expectation Maximization
(EM) algorithm, are notoriously sensitive to parameter initialization and often
suffer from bad local optima that could be arbitrarily worse than the optimal.
To address the long-lasting bad-local-optima challenge, we draw inspiration
from the recent ground-breaking foundation models and propose to leverage their
underlying big learning principle to upgrade the EM. Specifically, we present
the Big Learning EM (BigLearn-EM), an EM upgrade that simultaneously performs
joint, marginal, and orthogonally transformed marginal matchings between data
and model distributions. Through simulated experiments, we empirically show
that the BigLearn-EM is capable of delivering the optimal with high
probability; comparisons on benchmark clustering datasets further demonstrate
its effectiveness and advantages over existing techniques. The code is
available at
this https URL

提出了一种名为 BigLearn-EM 的算法，通过模拟实验证明了其在处理混合模型中的局部最优问题方面的有效性和优势。

大学习期望最大化

Big Learning Expectation Maximization

Matrix completion is a well-studied problem with many machine learning
applications. In practice, the problem is often solved by non-convex
optimization algorithms. However, the current theoretical analysis for
non-convex algorithms relies heavily on the assumption that every entry is
observed with exactly the same probability $p$, which is not realistic in
practice.
In this paper, we investigate a more realistic semi-random model, where the
probability of observing each entry is at least $p$. Even with this mild
semi-random perturbation, we can construct counter-examples where existing
non-convex algorithms get stuck in bad local optima.
In light of the negative results, we propose a pre-processing step that tries
to re-weight the semi-random input, so that it becomes "similar" to a random
input. We give a nearly-linear time algorithm for this problem, and show that
after our pre-processing, all the local minima of the non-convex objective can
be used to approximately recover the underlying ground-truth matrix.

本文提出在非凸优化算法的研究中，矩阵补全问题是一个重要的问题，但现有的理论分析都基于每个条目都有相同的概率被观察到的假设，不符合实际。因此，当每个条目被观察到的概率稍有不同的情况下，我们提出一种预处理方法，以便可以恢复基础地真实矩阵。