We show that the gradient estimates used in training Deep Gaussian Processes (DGPs) with importance-weighted variational inference are susceptible to signal-to-noise ratio (SNR) issues. Specifically, we show both theoretically and empirically that the SNR of the gradient estimates for the latent variable's variational parameters decreases as the number of importance samples increases. As a result, these gradient estimates degrade to pure noise if the number of importance samples is too large. To address this pathology, we show how doubly-reparameterized gradient estimators, originally proposed for training variational autoencoders, can be adapted to the DGP setting and that the resultant estimators completely remedy the SNR issue, thereby providing more reliable training. Finally, we demonstrate that our fix can lead to improvements in the predictive performance of the model's predictive posterior.

通过理论分析和大量实验证明，在Deep Gaussian Processes (DGPs)使用加权变分推断训练的梯度估计中，信噪比问题是普遍存在的。本文提出的基于双重重参数化的梯度估计方法解决了这个问题，提高了DGPs模型的预测性能。

关于深高斯过程变分推断中信噪比问题的研究