Variational inference (VI) is a popular approach in Bayesian inference, that
looks for the best approximation of the posterior distribution within a
parametric family, minimizing a loss that is typically the (reverse)
Kullback-Leibler (KL) divergence. Despite its empirical success, the
theoretical properties of VI have only received attention recently, and mostly
when the parametric family is the one of Gaussians. This work aims to
contribute to the theoretical study of VI in the non-Gaussian case by
investigating the setting of Mixture of Gaussians with fixed covariance and
constant weights. In this view, VI over this specific family can be casted as
the minimization of a Mollified relative entropy, i.e. the KL between the
convolution (with respect to a Gaussian kernel) of an atomic measure supported
on Diracs, and the target distribution. The support of the atomic measure
corresponds to the localization of the Gaussian components. Hence, solving
variational inference becomes equivalent to optimizing the positions of the
Diracs (the particles), which can be done through gradient descent and takes
the form of an interacting particle system. We study two sources of error of
variational inference in this context when optimizing the mollified relative
entropy. The first one is an optimization result, that is a descent lemma
establishing that the algorithm decreases the objective at each iteration. The
second one is an approximation error, that upper bounds the objective between
an optimal finite mixture and the target distribution.

通过优化固定协方差和常值权重的高斯混合模型，将变分推断（Variational Inference）视为最小化平滑相对熵，研究其在非高斯情况下的理论性质，包括梯度下降和粒子系统优化。