The mean field methods, which entail approximating intractable probability distributions variationally with distributions from a tractable family, enjoy high efficiency, guaranteed convergence, and provide lower bounds on the true likelihood. But due to requirement for model-specific derivation of the optimization equations and unclear inference quality in various models, it is not widely used as a generic approximate inference algorithm. In this paper, we discuss a generalized mean field theory on variational approximation to a broad class of intractable distributions using a rich set of tractable distributions via constrained optimization over distribution spaces. We present a class of generalized mean field (GMF) algorithms for approximate inference in complex exponential family models, which entails limiting the optimization over the class of cluster-factorizable distributions. GMF is a generic method requiring no model-specific derivations. It factors a complex model into a set of disjoint variable clusters, and uses a set of canonical fix-point equations to iteratively update the cluster distributions, and converge to locally optimal cluster marginals that preserve the original dependency structure within each cluster, hence, fully decomposed the overall inference problem. We empirically analyzed the effect of different tractable family (clusters of different granularity) on inference quality, and compared GMF with BP on several canonical models. Possible extension to higher-order MF approximation is also discussed.

在这篇论文中，我们讨论了一种广义均场理论，它使用可追踪分布的丰富集合来近似表示一类难以处理的分布，并通过分布空间上的约束优化来实现。我们还介绍了一类广义均场算法，用于复杂的指数族模型的近似推断，它将复杂模型分解成一组不相交变量簇，并使用一组规范的定点方程来迭代地更新参数，找到保留每个簇内原始依赖结构的局部最优参数，因此完全分解了整体推断问题。通过实验证明，我们比较了不同可追踪族群（不同粒度的簇）对推理质量的影响，并将其与BP算法在几个标准模型上进行了比较。还讨论了高阶MF逼近的可能扩展。

一种泛化均值场算法用于指数族的变分推断