As diffusion probabilistic models (DPMs) are being employed as mainstream
models for generative artificial intelligence (AI), the study of their
memorization of the raw training data has attracted growing attention. Existing
works in this direction aim to establish an understanding of whether or to what
extent DPMs learn by memorization. Such an understanding is crucial for
identifying potential risks of data leakage and copyright infringement in
diffusion models and, more importantly, for more controllable generation and
trustworthy application of Artificial Intelligence Generated Content (AIGC).
While previous works have made important observations of when DPMs are prone to
memorization, these findings are mostly empirical, and the developed data
extraction methods only work for conditional diffusion models. In this work, we
aim to establish a theoretical understanding of memorization in DPMs with 1) a
memorization metric for theoretical analysis, 2) an analysis of conditional
memorization with informative and random labels, and 3) two better evaluation
metrics for measuring memorization. Based on the theoretical analysis, we
further propose a novel data extraction method called \textbf{Surrogate
condItional Data Extraction (SIDE)} that leverages a classifier trained on
generated data as a surrogate condition to extract training data directly from
unconditional diffusion models. Our empirical results demonstrate that SIDE can
extract training data from diffusion models where previous methods fail, and it
is on average over 50\% more effective across different scales of the CelebA
dataset.

扩散概率模型的记忆化、理论分析、条件记忆化、数据提取、Surrogate condItional Data Extraction (SIDE)