Diffusion models (DMs) demonstrate potent image generation capabilities in
various generative modeling tasks. Nevertheless, their primary limitation lies
in slow sampling speed, requiring hundreds or thousands of sequential function
evaluations through large neural networks to generate high-quality images.
Sampling from DMs can be seen as solving corresponding stochastic differential
equations (SDEs) or ordinary differential equations (ODEs). In this work, we
formulate the sampling process as an extended reverse-time SDE (ER SDE),
unifying prior explorations into ODEs and SDEs. Leveraging the semi-linear
structure of ER SDE solutions, we offer exact solutions and arbitrarily
high-order approximate solutions for VP SDE and VE SDE, respectively. Based on
the solution space of the ER SDE, we yield mathematical insights elucidating
the superior performance of ODE solvers over SDE solvers in terms of fast
sampling. Additionally, we unveil that VP SDE solvers stand on par with their
VE SDE counterparts. Finally, we devise fast and training-free samplers, ER-SDE
Solvers, elevating the efficiency of stochastic samplers to unprecedented
levels. Experimental results demonstrate achieving 3.45 FID in 20 function
evaluations and 2.24 FID in 50 function evaluations on the ImageNet
64$\times$64 dataset.

通过将采样过程定义为扩展的逆时间随机微分方程 (ERSDE)，我们提出了 ER-SDE 求解器，为扩散模型 (DM) 的图像生成速度带来了突破性的提升，并在 ImageNet 64×64 数据集上实现了 3.45 FID 的效果。

扩展逆时间 SDE 扩散模型解空间的阐明

Elucidating the solution space of extended reverse-time SDE for  diffusion models

A potent class of generative models known as Diffusion Probabilistic Models
(DPMs) has become prominent. A forward diffusion process adds gradually noise
to data, while a model learns to gradually denoise. Sampling from pre-trained
DPMs is obtained by solving differential equations (DE) defined by the learnt
model, a process which has shown to be prohibitively slow. Numerous efforts on
speeding-up this process have consisted on crafting powerful ODE solvers.
Despite being quick, such solvers do not usually reach the optimal quality
achieved by available slow SDE solvers. Our goal is to propose SDE solvers that
reach optimal quality without requiring several hundreds or thousands of NFEs
to achieve that goal. In this work, we propose Stochastic Exponential
Derivative-free Solvers (SEEDS), improving and generalizing Exponential
Integrator approaches to the stochastic case on several frameworks. After
carefully analyzing the formulation of exact solutions of diffusion SDEs, we
craft SEEDS to analytically compute the linear part of such solutions. Inspired
by the Exponential Time-Differencing method, SEEDS uses a novel treatment of
the stochastic components of solutions, enabling the analytical computation of
their variance, and contains high-order terms allowing to reach optimal quality
sampling $\sim3$-$5\times$ faster than previous SDE methods. We validate our
approach on several image generation benchmarks, showing that SEEDS outperforms
or is competitive with previous SDE solvers. Contrary to the latter, SEEDS are
derivative and training free, and we fully prove strong convergence guarantees
for them.

本文提出了 SEEDS 方法，用于快速且精确地从 DPMs 中采样，在多个框架中改进和推广指数积分器方法，引入高阶项，使其比以前的 SDE 方法快 3-5 倍，在图像生成基准测试中验证了该方法的性能。

SEEDS: 基于指数随机微分方程的快速高质量扰动模型采样器

SEEDS: Exponential SDE Solvers for Fast High-Quality Sampling from  Diffusion Models

Score-based (denoising diffusion) generative models have recently gained a
lot of success in generating realistic and diverse data. These approaches
define a forward diffusion process for transforming data to noise and generate
data by reversing it (thereby going from noise to data). Unfortunately, current
score-based models generate data very slowly due to the sheer number of score
network evaluations required by numerical SDE solvers.
In this work, we aim to accelerate this process by devising a more efficient
SDE solver. Existing approaches rely on the Euler-Maruyama (EM) solver, which
uses a fixed step size. We found that naively replacing it with other SDE
solvers fares poorly - they either result in low-quality samples or become
slower than EM. To get around this issue, we carefully devise an SDE solver
with adaptive step sizes tailored to score-based generative models piece by
piece. Our solver requires only two score function evaluations, rarely rejects
samples, and leads to high-quality samples. Our approach generates data 2 to 10
times faster than EM while achieving better or equal sample quality. For
high-resolution images, our method leads to significantly higher quality
samples than all other methods tested. Our SDE solver has the benefit of
requiring no step size tuning.

本文提出一种利用自适应步长的随机微分方程求解器来加速基于评分的生成模型，方法能够稳定高效地生成高质量的数据，特别是在高分辨率图像生成方面表现优异。