Posterior collapse in Variational Autoencoders (VAEs) arises when the variational posterior distribution closely matches the prior for a subset of latent variables. This paper presents a simple and intuitive explanation for posterior collapse through the analysis of linear VAEs and their direct correspondence with Probabilistic PCA (pPCA). We explain how posterior collapse may occur in pPCA due to local maxima in the log marginal likelihood. Unexpectedly, we prove that the ELBO objective for the linear VAE does not introduce additional spurious local maxima relative to log marginal likelihood. We show further that training a linear VAE with exact variational inference recovers an identifiable global maximum corresponding to the principal component directions. Empirically, we find that our linear analysis is predictive even for high-capacity, non-linear VAEs and helps explain the relationship between the observation noise, local maxima, and posterior collapse in deep Gaussian VAEs.

本文通过对线性变分自编码器和概率PCA之间的直接对应关系进行分析，提出了有关后验崩溃的简单而直观的解释。从计算角度出发证明了线性VAE的ELBO目标不会引入附加的虚假局部极值，进一步证明了使用确切变分推理培训线性VAE可以恢复对应于主成分方向的全局最大值。同时，我们的线性分析对于高容量的非线性VAEs具有预测性，并有助于解释观察噪声，局部最大值和后验崩溃之间的关系。

不要怪ELBO！线性VAE视角下的后验崩溃问题