Semi-implicit variational inference (SIVI) extends traditional variational
families with semi-implicit distributions defined in a hierarchical manner. Due
to the intractable densities of semi-implicit distributions, classical SIVI
often resorts to surrogates of evidence lower bound (ELBO) that would introduce
biases for training. A recent advancement in SIVI, named SIVI-SM, utilizes an
alternative score matching objective made tractable via a minimax formulation,
albeit requiring an additional lower-level optimization. In this paper, we
propose kernel SIVI (KSIVI), a variant of SIVI-SM that eliminates the need for
lower-level optimization through kernel tricks. Specifically, we show that when
optimizing over a reproducing kernel Hilbert space (RKHS), the lower-level
problem has an explicit solution. This way, the upper-level objective becomes
the kernel Stein discrepancy (KSD), which is readily computable for stochastic
gradient descent due to the hierarchical structure of semi-implicit variational
distributions. An upper bound for the variance of the Monte Carlo gradient
estimators of the KSD objective is derived, which allows us to establish novel
convergence guarantees of KSIVI. We demonstrate the effectiveness and
efficiency of KSIVI on both synthetic distributions and a variety of real data
Bayesian inference tasks.

核 SIVI（KSIVI）是 SIVI-SM 的变体，通过核技巧消除了对底层优化的需求，其上层目标是可计算的核 Stein 距离（KSD），从而实现了 KSIVI 的新收敛保证。我们在合成分布和各种真实数据贝叶斯推断任务中展示了 KSIVI 的有效性和效率。

核半隐式变分推断

Kernel Semi-Implicit Variational Inference

Semi-implicit variational inference (SIVI) greatly enriches the
expressiveness of variational families by considering implicit variational
distributions defined in a hierarchical manner. However, due to the intractable
densities of variational distributions, current SIVI approaches often use
surrogate evidence lower bounds (ELBOs) or employ expensive inner-loop MCMC
runs for unbiased ELBOs for training. In this paper, we propose SIVI-SM, a new
method for SIVI based on an alternative training objective via score matching.
Leveraging the hierarchical structure of semi-implicit variational families,
the score matching objective allows a minimax formulation where the intractable
variational densities can be naturally handled with denoising score matching.
We show that SIVI-SM closely matches the accuracy of MCMC and outperforms
ELBO-based SIVI methods in a variety of Bayesian inference tasks.

SIVI-SM 是基于分数匹配的新的 SIVI 方法，通过利用半隐变分族的分层结构，允许使用少噪声分数匹配处理难以处理的变分密度，SIVI-SM 在各种贝叶斯推理任务中准确性与 MCMC 相当，并且优于基于 ELBO 的 SIVI 方法。