The ability to backpropagate stochastic gradients through continuous latent
distributions has been crucial to the emergence of variational autoencoders and
stochastic gradient variational Bayes. The key ingredient is an unbiased and
low-variance way of estimating gradients with respect to distribution
parameters from gradients evaluated at distribution samples. The
"reparameterization trick" provides a class of transforms yielding such
estimators for many continuous distributions, including the Gaussian and other
members of the location-scale family. However the trick does not readily extend
to mixture density models, due to the difficulty of reparameterizing the
discrete distribution over mixture weights. This report describes an
alternative transform, applicable to any continuous multivariate distribution
with a differentiable density function from which samples can be drawn, and
uses it to derive an unbiased estimator for mixture density weight derivatives.
Combined with the reparameterization trick applied to the individual mixture
components, this estimator makes it straightforward to train variational
autoencoders with mixture-distributed latent variables, or to perform
stochastic variational inference with a mixture density variational posterior.

通过 reparameterization trick 和新的变换方法，本文提出了一种有效的估算混合密度模型中梯度的方法，可以用于训练具有混合分布潜变量的变分自编码器或执行混合密度变分后验的随机变分推断。