Variational autoencoders (VAE) are directed generative models that learn factorial latent variables. As noted by Burda et al. (2015), these models exhibit the problem of factor over-pruning where a significant number of stochastic factors fail to learn anything and become inactive. This can limit their modeling power and their ability to learn diverse and meaningful latent representations. In this paper, we evaluate several methods to address this problem and propose a more effective model-based approach called the epitomic variational autoencoder (eVAE). The so-called epitomes of this model are groups of mutually exclusive latent factors that compete to explain the data. This approach helps prevent inactive units since each group is pressured to explain the data. We compare the approaches with qualitative and quantitative results on MNIST and TFD datasets. Our results show that eVAE makes efficient use of model capacity and generalizes better than VAE.

本研究比较了多种方法来解决变分自编码器中随机因素无法学习和失活等问题，并提出了一种更有效的基于模型的方法，称为最佳变分自编码器，该方法有助于防止单元失活，并比VAE更好地使用了模型容量，产生更好的泛化性能。

解决变分自编码器中的过度剪枝问题