Variational autoencoders (VAEs) face a notorious problem wherein the variational posterior often aligns closely with the prior, a phenomenon known as posterior collapse, which hinders the quality of representation learning. To mitigate this problem, an adjustable hyperparameter $\beta$ and a strategy for annealing this parameter, called KL annealing, are proposed. This study presents a theoretical analysis of the learning dynamics in a minimal VAE. It is rigorously proved that the dynamics converge to a deterministic process within the limit of large input dimensions, thereby enabling a detailed dynamical analysis of the generalization error. Furthermore, the analysis shows that the VAE initially learns entangled representations and gradually acquires disentangled representations. A fixed-point analysis of the deterministic process reveals that when $\beta$ exceeds a certain threshold, posterior collapse becomes inevitable regardless of the learning period. Additionally, the superfluous latent variables for the data-generative factors lead to overfitting of the background noise; this adversely affects both generalization and learning convergence. The analysis further unveiled that appropriately tuned KL annealing can accelerate convergence.

在这项研究中，对变分自编码器（VAEs）中的后验折叠问题进行了理论分析，发现学习动态会在输入维度趋近无限大时收敛为确定性过程。此外，研究还表明VAE最初学习到纠缠表示，并逐渐获得解耦表示。当超参数β超过某一阈值时，无论学习周期如何，后验折叠都是不可避免的。背景噪声引发了超量潜在变量，导致过拟合，损害了泛化和学习收敛性。研究还发现适当调整的KL渐近算法能加速收敛。

线性VAE中的学习动力学：后验崩塌临界点、多余潜变空间陷阱和KL退火加速