Recently, diffusion models have emerged as powerful deep generative models,
showcasing cutting-edge performance across various applications such as image
generation, solving inverse problems, and text-to-image synthesis. These models
generate new data (e.g., images) by transforming random noise inputs through a
reverse diffusion process. In this work, we uncover a distinct and prevalent
phenomenon within diffusion models in contrast to most other generative models,
which we refer to as ``consistent model reproducibility''. To elaborate, our
extensive experiments have consistently shown that when starting with the same
initial noise input and sampling with a deterministic solver, diffusion models
tend to produce nearly identical output content. This consistency holds true
regardless of the choices of model architectures and training procedures.
Additionally, our research has unveiled that this exceptional model
reproducibility manifests in two distinct training regimes: (i) ``memorization
regime,'' characterized by a significantly overparameterized model which
attains reproducibility mainly by memorizing the training data; (ii)
``generalization regime,'' in which the model is trained on an extensive
dataset, and its reproducibility emerges with the model's generalization
capabilities. Our analysis provides theoretical justification for the model
reproducibility in ``memorization regime''. Moreover, our research reveals that
this valuable property generalizes to many variants of diffusion models,
including conditional diffusion models, diffusion models for solving inverse
problems, and fine-tuned diffusion models. A deeper understanding of this
phenomenon has the potential to yield more interpretable and controllable data
generative processes based on diffusion models.

扩散模型在图像生成、解决逆问题和文本到图像合成等各种应用中展示了领先的性能。我们揭示了与大多数其他生成模型截然不同的 “一致模型可重现性” 现象，通过广泛的实验表明，扩散模型在相同初始噪声输入和确定性求解器采样时，倾向于产生几乎相同的输出内容。此模型可重现性在不同的训练环境下均成立，包括记忆化和泛化模式。进一步的分析提供了对 “记忆化模式” 中模型可重现性的理论解释，并揭示此有价值的特性适用于许多扩散模型的变种。更深入理解此现象有可能产生基于扩散模型的更可解释和可控的数据生成过程。