As deep generative models have progressed, recent work has shown them to be capable of memorizing and reproducing training datapoints when deployed. These findings call into question the usability of generative models, especially in light of the legal and privacy risks brought about by memorization. To better understand this phenomenon, we propose the manifold memorization hypothesis (MMH), a geometric framework which leverages the manifold hypothesis into a clear language in which to reason about memorization. We propose to analyze memorization in terms of the relationship between the dimensionalities of $(i)$ the ground truth data manifold and $(ii)$ the manifold learned by the model. This framework provides a formal standard for "how memorized" a datapoint is and systematically categorizes memorized data into two types: memorization driven by overfitting and memorization driven by the underlying data distribution. By analyzing prior work in the context of the MMH, we explain and unify assorted observations in the literature. We empirically validate the MMH using synthetic data and image datasets up to the scale of Stable Diffusion, developing new tools for detecting and preventing generation of memorized samples in the process.

本研究旨在解决深度生成模型在训练过程中记忆和再现数据点的问题，特别是在法律和隐私风险方面的关注。提出的流形记忆假设（MMH）提供了一个几何框架，从流形维度的关系分析记忆现象，系统分类记忆数据的类型，并经过实验证明该框架有效性，推动生成模型在记忆样本检测和防止方面的新工具开发。

理解生成模型中的记忆的几何框架