We propose a simple and general variant of the standard reparameterized
gradient estimator for the variational evidence lower bound. Specifically, we
remove a part of the total derivative with respect to the variational
parameters that corresponds to the score function. Removing this term produces
an unbiased gradient estimator whose variance approaches zero as the
approximate posterior approaches the exact posterior. We analyze the behavior
of this gradient estimator theoretically and empirically, and generalize it to
more complex variational distributions such as mixtures and importance-weighted
posteriors.

我们提出了一种简单且通用的标准重参数化梯度估计变体，以用于变分证据下限。通过删除与评估参数有关的分数函数的导数，我们将产生一个无偏梯度估计器，其方差随着近似后验接近精确后验逐渐逼近零。我们从理论和实证方面分析了这种梯度估计器的行为，并将其推广到更复杂的变分分布中，例如混合分布和重要性加权后验。

降落完毕：变分推断中的简单、低差异度梯度估计器

Sticking the Landing: Simple, Lower-Variance Gradient Estimators for  Variational Inference

Tensor factorization is a powerful tool to analyse multi-way data. Compared
with traditional multi-linear methods, nonlinear tensor factorization models
are capable of capturing more complex relationships in the data. However, they
are computationally expensive and may suffer severe learning bias in case of
extreme data sparsity. To overcome these limitations, in this paper we propose
a distributed, flexible nonlinear tensor factorization model. Our model can
effectively avoid the expensive computations and structural restrictions of the
Kronecker-product in existing TGP formulations, allowing an arbitrary subset of
tensorial entries to be selected to contribute to the training. At the same
time, we derive a tractable and tight variational evidence lower bound (ELBO)
that enables highly decoupled, parallel computations and high-quality
inference. Based on the new bound, we develop a distributed inference algorithm
in the MapReduce framework, which is key-value-free and can fully exploit the
memory cache mechanism in fast MapReduce systems such as SPARK. Experimental
results fully demonstrate the advantages of our method over several
state-of-the-art approaches, in terms of both predictive performance and
computational efficiency. Moreover, our approach shows a promising potential in
the application of Click-Through-Rate (CTR) prediction for online advertising.

本文提出了一种分布式、灵活的非线性张量分解模型，通过可避免昂贵的计算以及提供高质量推理的上限，它能够克服传统张量分解模型中的限制，并展现出在 CTR 预测方面的巨大潜力。