This work presents an analysis of the hidden representations of Variational Autoencoders (VAEs) using the Intrinsic Dimension (ID) and the Information Imbalance (II). We show that VAEs undergo a transition in behaviour once the bottleneck size is larger than the ID of the data, manifesting in a double hunchback ID profile and a qualitative shift in information processing as captured by the II. Our results also highlight two distinct training phases for architectures with sufficiently large bottleneck sizes, consisting of a rapid fit and a slower generalisation, as assessed by a differentiated behaviour of ID, II, and KL loss. These insights demonstrate that II and ID could be valuable tools for aiding architecture search, for diagnosing underfitting in VAEs, and, more broadly, they contribute to advancing a unified understanding of deep generative models through geometric analysis.

该研究解决了变分自编码器（VAE）隐藏表示分析中的一个问题，通过内在维度（ID）和信息不平衡（II）提供了新的理解方法。研究发现，当瓶颈大小超过数据的内在维度时，VAE的行为会发生变化，并且确认了具有较大瓶颈的结构存在快速拟合和缓慢泛化的两个独特训练阶段，这为优化架构搜索、诊断VAE的欠拟合问题提供了有价值的工具，并推动了对深度生成模型的统一理解。

理解变分自编码器的内在维度与信息不平衡