Score-based generative models (SGMs) have revolutionized the field of generative modeling, achieving unprecedented success in generating realistic and diverse content. Despite empirical advances, the theoretical basis for why optimizing the evidence lower bound (ELBO) on the log-likelihood is effective for training diffusion generative models, such as DDPMs, remains largely unexplored. In this paper, we address this question by establishing a density formula for a continuous-time diffusion process, which can be viewed as the continuous-time limit of the forward process in an SGM. This formula reveals the connection between the target density and the score function associated with each step of the forward process. Building on this, we demonstrate that the minimizer of the optimization objective for training DDPMs nearly coincides with that of the true objective, providing a theoretical foundation for optimizing DDPMs using the ELBO. Furthermore, we offer new insights into the role of score-matching regularization in training GANs, the use of ELBO in diffusion classifiers, and the recently proposed diffusion loss.

本文解决了优化证据下界（ELBO）对扩散生成模型（如DDPMs）训练有效性的理论基础问题，建立了连续时间扩散过程的密度公式，并揭示了目标密度与每一步得分函数之间的联系。研究发现，训练DDPMs的优化目标的最小化几乎与真实目标一致，为使用ELBO优化DDPMs提供了理论支持，同时为GAN培训中的得分匹配正则化及扩散分类器的ELBO使用提供了新的见解。

基于得分的密度公式及其在扩散生成模型中的应用