The diffusion probabilistic generative models are widely used to generate
high-quality data. Though they can synthetic data that does not exist in the
training set, the rationale behind such generalization is still unexplored. In
this paper, we formally define the generalization of the generative model,
which is measured by the mutual information between the generated data and the
training set. The definition originates from the intuition that the model which
generates data with less correlation to the training set exhibits better
generalization ability. Meanwhile, we show that for the empirical optimal
diffusion model, the data generated by a deterministic sampler are all highly
related to the training set, thus poor generalization. This result contradicts
the observation of the trained diffusion model's (approximating empirical
optima) extrapolation ability (generating unseen data). To understand this
contradiction, we empirically verify the difference between the sufficiently
trained diffusion model and the empirical optima. We found, though obtained
through sufficient training, there still exists a slight difference between
them, which is critical to making the diffusion model generalizable. Moreover,
we propose another training objective whose empirical optimal solution has no
potential generalization problem. We empirically show that the proposed
training objective returns a similar model to the original one, which further
verifies the generalization ability of the trained diffusion model.

通过定义通用性测量，本文研究了扩散概率生成模型的泛化能力。我们发现确定性采样器生成的数据与训练集高度相关，因此泛化能力差。通过实验，我们验证了足够训练的扩散模型与经验最优解之间的微小区别对于模型具有关键性影响，并提出了另一个无潜在泛化问题的训练目标，该目标返回与原模型类似的模型，从而验证了训练扩散模型的泛化能力。