Stein Variational Gradient Descent (SVGD) is a popular particle-based method for Bayesian inference. However, its convergence suffers from the variance collapse, which reduces the accuracy and diversity of the estimation. In this paper, we study the isotropy property of finite particles during the convergence process and show that SVGD of finite particles cannot spread across the entire sample space. Instead, all particles tend to cluster around the particle center within a certain range and we provide an analytical bound for this cluster. To further improve the effectiveness of SVGD for high-dimensional problems, we propose the Augmented Message Passing SVGD (AUMP-SVGD) method, which is a two-stage optimization procedure that does not require sparsity of the target distribution, unlike the MP-SVGD method. Our algorithm achieves satisfactory accuracy and overcomes the variance collapse problem in various benchmark problems.

本文研究了Stein Variational Gradient Descent（SVGD）方法收敛过程中粒子的各向同性特性，并证明了有限粒子的SVGD无法扩展到整个样本空间，而是倾向于在一定范围内围绕粒子中心聚集；我们提出了一种改进的SVGD方法——增强式信息传递SVGD（AUMP-SVGD），该方法是一个不需要目标分布稀疏性的两阶段优化过程，能够在各种基准问题中高效地解决方差崩溃问题，实验结果表明此方法能够取得令人满意的精度。

增强型消息传递 Stein 变分梯度下降