Variational families with full-rank covariance approximations are known not
to work well in black-box variational inference (BBVI), both empirically and
theoretically. In fact, recent computational complexity results for BBVI have
established that full-rank variational families scale poorly with the
dimensionality of the problem compared to e.g. mean field families. This is
particularly critical to hierarchical Bayesian models with local variables;
their dimensionality increases with the size of the datasets. Consequently, one
gets an iteration complexity with an explicit $\mathcal{O}(N^2)$ dependence on
the dataset size $N$. In this paper, we explore a theoretical middle ground
between mean-field variational families and full-rank families: structured
variational families. We rigorously prove that certain scale matrix structures
can achieve a better iteration complexity of $\mathcal{O}(N)$, implying better
scaling with respect to $N$. We empirically verify our theoretical results on
large-scale hierarchical models.

给定的研究论文提出了一个介于均场变分族和完全秩变分族之间的结构变分族，通过理论证明某些规模矩阵结构可以获得更好的迭代复杂度，并在大规模分层模型上进行实证验证。

可证明可扩展的结构化变分家族的黑盒变分推理

Provably Scalable Black-Box Variational Inference with Structured  Variational Families

Semi-implicit variational inference (SIVI) greatly enriches the
expressiveness of variational families by considering implicit variational
distributions defined in a hierarchical manner. However, due to the intractable
densities of variational distributions, current SIVI approaches often use
surrogate evidence lower bounds (ELBOs) or employ expensive inner-loop MCMC
runs for unbiased ELBOs for training. In this paper, we propose SIVI-SM, a new
method for SIVI based on an alternative training objective via score matching.
Leveraging the hierarchical structure of semi-implicit variational families,
the score matching objective allows a minimax formulation where the intractable
variational densities can be naturally handled with denoising score matching.
We show that SIVI-SM closely matches the accuracy of MCMC and outperforms
ELBO-based SIVI methods in a variety of Bayesian inference tasks.

SIVI-SM 是基于分数匹配的新的 SIVI 方法，通过利用半隐变分族的分层结构，允许使用少噪声分数匹配处理难以处理的变分密度，SIVI-SM 在各种贝叶斯推理任务中准确性与 MCMC 相当，并且优于基于 ELBO 的 SIVI 方法。