Denoising autoencoders (DAE) are trained to reconstruct their clean input with noise injected at the input level, while variational autoencoders (VAE) are trained with noise injected in their stochastic hidden layer, with a regularizer that encourages this noise injection. In this paper, we show that injecting noise both in input and in the stochastic hidden layer can be advantageous and we propose a modified variational lower bound as an improved objective function in this setup. If noise is injected in input, then the standard VAE lower bound involves marginalizing the encoder conditional distribution over the input noise, which makes the training criterion intractable. Instead, we propose a modified training criterion which corresponds to a tighter bound, when noise is injected in input. Experimentally, we find that the proposed denoising variational autoencoder (DVAE) yields better average log-likelihood than the VAE and the importance weighted auto-encoder on the MNIST and Frey Face datasets.

本文研究了对输入和隐层同时进行噪声注入的变分自编码器，提出了一种改进的目标函数。当输入数据有噪声时，传统的变分自编码器的训练方法不可行，这里提出了一种可行的训练方法。实验结果表明，在MNIST和Frey Face数据集上，提出的去噪变分自编码器（DVAE）的平均对数似然比VAE和重要性加权自编码器更好。

变分自编码框架中的去噪准则