Despite their success in real data synthesis, diffusion models (DMs) often
suffer from slow and costly training and sampling issues, limiting their
broader applications. To mitigate this, we propose a Fast Diffusion Model (FDM)
which improves the diffusion process of DMs from a stochastic optimization
perspective to speed up both training and sampling. Specifically, we first find
that the diffusion process of DMs accords with the stochastic optimization
process of stochastic gradient descent (SGD) on a stochastic time-variant
problem. Note that momentum SGD uses both the current gradient and an extra
momentum, achieving more stable and faster convergence. We are inspired to
introduce momentum into the diffusion process to accelerate both training and
sampling. However, this comes with the challenge of deriving the noise
perturbation kernel from the momentum-based diffusion process. To this end, we
frame the momentum-based process as a Damped Oscillation system whose
critically damped state -- the kernel solution -- avoids oscillation and thus
has a faster convergence speed of the diffusion process. Empirical results show
that our FDM can be applied to several popular DM frameworks, e.g. VP, VE, and
EDM, and reduces their training cost by about 50% with comparable image
synthesis performance on CIFAR-10, FFHQ, and AFHQv2 datasets. Moreover, FDM
decreases their sampling steps by about $3\times$ to achieve similar
performance under the same deterministic samplers. The code is available at
this https URL

提出了快速扩散模型（FDM），它将扩散模型（DM）的扩散过程从随机优化角度进行改进，以加速训练和采样。实验证明，FDM 可以应用于多种流行的 DM 框架，并在 CIFAR-10、FFHQ 和 AFHQv2 数据集上具有可比的图像合成性能。而且，FDM 通过将采样步骤减少约 3 倍来实现相似的性能，从而将训练成本降低约 50％。