Deep generative models have made tremendous progress in modeling complex
data, often exhibiting generation quality that surpasses a typical human's
ability to discern the authenticity of samples. Undeniably, a key driver of
this success is enabled by the massive amounts of web-scale data consumed by
these models. Due to these models' striking performance and ease of
availability, the web will inevitably be increasingly populated with synthetic
content. Such a fact directly implies that future iterations of generative
models must contend with the reality that their training is curated from both
clean data and artificially generated data from past models. In this paper, we
develop a framework to rigorously study the impact of training generative
models on mixed datasets (of real and synthetic data) on their stability. We
first prove the stability of iterative training under the condition that the
initial generative models approximate the data distribution well enough and the
proportion of clean training data (w.r.t. synthetic data) is large enough. We
empirically validate our theory on both synthetic and natural images by
iteratively training normalizing flows and state-of-the-art diffusion models on
CIFAR10 and FFHQ.

本文开发了一个框架，以严格研究在混合数据集（包括真实和合成数据）上训练生成模型对它们的稳定性的影响。通过在 CIFAR10 和 FFHQ 上迭代训练常规流和最先进的扩散模型，我们在合成和自然图像上经验验证了我们的理论。