Automatic Differentiation Variational Inference (ADVI) is efficient in
learning probabilistic models. Classic ADVI relies on the parametric approach
to approximate the posterior. In this paper, we develop a spline-based
nonparametric approximation approach that enables flexible posterior
approximation for distributions with complicated structures, such as skewness,
multimodality, and bounded support. Compared with widely-used nonparametric
variational inference methods, the proposed method is easy to implement and
adaptive to various data structures. By adopting the spline approximation, we
derive a lower bound of the importance weighted autoencoder and establish the
asymptotic consistency. Experiments demonstrate the efficiency of the proposed
method in approximating complex posterior distributions and improving the
performance of generative models with incomplete data.

我们提出了一种基于样条的非参数逼近方法，可以灵活逼近具有复杂结构的后验分布，如偏度、多峰性和有界支持。通过采用样条逼近，我们得到了重要性加权自编码器的下界，并建立了渐近一致性。实验证明了该方法在逼近复杂的后验分布和提高具有不完整数据的生成模型性能方面的高效性。

非参数自动微分变分推断与样条逼近

Nonparametric Automatic Differentiation Variational Inference with  Spline Approximation

The denoising diffusion model emerges recently as a powerful generative
technique that converts noise into data. Theoretical convergence guarantee has
been mainly studied for continuous-time diffusion models, and has been obtained
for discrete-time diffusion models only for distributions with bounded support
in the literature. In this paper, we establish the convergence guarantee for
substantially larger classes of distributions under discrete-time diffusion
models and further improve the convergence rate for distributions with bounded
support. In particular, we first establish the convergence rates for both
smooth and general (possibly non-smooth) distributions having finite second
moment. We then specialize our results to a number of interesting classes of
distributions with explicit parameter dependencies, including distributions
with Lipschitz scores, Gaussian mixture distributions, and distributions with
bounded support. We further propose a novel accelerated sampler and show that
it improves the convergence rates of the corresponding regular sampler by
orders of magnitude with respect to all system parameters. For distributions
with bounded support, our result improves the dimensional dependence of the
previous convergence rate by orders of magnitude. Our study features a novel
analysis technique that constructs tilting factor representation of the
convergence error and exploits Tweedie's formula for handling Taylor expansion
power terms.

去噪扩散模型是一种将噪声转换为数据的强大生成技术，本论文研究了离散时间扩散模型在更大范围的分布上的收敛性保证，并提出了一种加速采样器来提高收敛速度和维度依赖性。