We introduce Score identity Distillation (SiD), an innovative data-free method that distills the generative capabilities of pretrained diffusion models into a single-step generator. SiD not only facilitates an exponentially fast reduction in Fr\'echet inception distance (FID) during distillation but also approaches or even exceeds the FID performance of the original teacher diffusion models. By reformulating forward diffusion processes as semi-implicit distributions, we leverage three score-related identities to create an innovative loss mechanism. This mechanism achieves rapid FID reduction by training the generator using its own synthesized images, eliminating the need for real data or reverse-diffusion-based generation, all accomplished within significantly shortened generation time. Upon evaluation across four benchmark datasets, the SiD algorithm demonstrates high iteration efficiency during distillation and surpasses competing distillation approaches, whether they are one-step or few-step, data-free, or dependent on training data, in terms of generation quality. This achievement not only redefines the benchmarks for efficiency and effectiveness in diffusion distillation but also in the broader field of diffusion-based generation. Our PyTorch implementation will be publicly accessible on GitHub.

我们引入了Score identity Distillation（SiD），一种创新的无数据方法，将预训练扩散模型的生成能力提炼到一个单步生成器中。通过将前向扩散过程重构为半隐式分布，利用三个与分数相关的身份，创造了一种创新的损失机制。这种机制通过使用自己合成的图像来训练生成器，在大大缩短的生成时间内消除了对真实数据或基于逆扩散的生成的需求，实现了快速的 Fréchet inception distance (FID) 减小。在四个基准数据集上评估后，SiD算法在蒸馏过程中显示出高迭代效率，并超越了在生成质量方面与其竞争的蒸馏方法，无论它们是一步或几步，无数据还是依赖于训练数据。这一成就不仅重新定义了扩散蒸馏中效率和效果的基准，还在更广泛的扩散生成领域中也有着重要的意义。我们的PyTorch实现将在GitHub上公开获取。

得分身份提炼：预先训练扩散模型的指数级快速提炼用于单步生成