Maximum likelihood estimation (MLE) of latent variable models is often recast
as an optimization problem over the extended space of parameters and
probability distributions. For example, the Expectation Maximization (EM)
algorithm can be interpreted as coordinate descent applied to a suitable free
energy functional over this space. Recently, this perspective has been combined
with insights from optimal transport and Wasserstein gradient flows to develop
particle-based algorithms applicable to wider classes of models than standard
EM.
Drawing inspiration from prior works which interpret `momentum-enriched'
optimisation algorithms as discretizations of ordinary differential equations,
we propose an analogous dynamical systems-inspired approach to minimizing the
free energy functional over the extended space of parameters and probability
distributions. The result is a dynamic system that blends elements of
Nesterov's Accelerated Gradient method, the underdamped Langevin diffusion, and
particle methods.
Under suitable assumptions, we establish quantitative convergence of the
proposed system to the unique minimiser of the functional in continuous time.
We then propose a numerical discretization of this system which enables its
application to parameter estimation in latent variable models. Through
numerical experiments, we demonstrate that the resulting algorithm converges
faster than existing methods and compares favourably with other (approximate)
MLE algorithms.

最大似然估计（MLE）的潜变量模型常常被重新设定为参数和概率分布的扩展空间上的优化问题。我们提出了一个受动力系统启发的方法，结合了 Nesterov 的加速梯度法、欠阻尼朗之万方程和粒子方法，使得该算法在连续时间中收敛到函数的唯一最小值。通过数值实验，我们证明该算法比现有方法更快地收敛，并与其他（近似的）MLE 算法相比具有优势。

动量粒子极大似然

Momentum Particle Maximum Likelihood

We introduce two new particle-based algorithms for learning latent variable
models via marginal maximum likelihood estimation, including one which is
entirely tuning-free. Our methods are based on the perspective of marginal
maximum likelihood estimation as an optimization problem: namely, as the
minimization of a free energy functional. One way to solve this problem is to
consider the discretization of a gradient flow associated with the free energy.
We study one such approach, which resembles an extension of the popular Stein
variational gradient descent algorithm. In particular, we establish a descent
lemma for this algorithm, which guarantees that the free energy decreases at
each iteration. This method, and any other obtained as the discretization of
the gradient flow, will necessarily depend on a learning rate which must be
carefully tuned by the practitioner in order to ensure convergence at a
suitable rate. With this in mind, we also propose another algorithm for
optimizing the free energy which is entirely learning rate free, based on coin
betting techniques from convex optimization. We validate the performance of our
algorithms across a broad range of numerical experiments, including several
high-dimensional settings. Our results are competitive with existing
particle-based methods, without the need for any hyperparameter tuning.

介绍了两种用于通过边缘最大似然估计学习潜变量模型的新的基于粒子的算法，其中包括一种完全免调范的算法。我们的方法基于作为优化问题的边缘最大似然估计的视角：即作为自由能泛函的最小化。我们验证了我们算法在广泛的数值试验中的性能，包括几个高维设置。