Significant advancements in image generation have been made with diffusion models. Nevertheless, when contrasted with previous generative models, diffusion models face substantial computational overhead, leading to failure in real-time generation. Recent approaches have aimed to accelerate diffusion models by reducing the number of sampling steps through improved sampling techniques or step distillation. However, the methods to diminish the computational cost for each timestep remain a relatively unexplored area. Observing the fact that diffusion models exhibit varying input distributions and feature distributions at different timesteps, we introduce one-to-many knowledge distillation (O2MKD), which distills a single teacher diffusion model into multiple student diffusion models, where each student diffusion model is trained to learn the teacher's knowledge for a subset of continuous timesteps. Experiments on CIFAR10, LSUN Church, CelebA-HQ with DDPM and COCO30K with Stable Diffusion show that O2MKD can be applied to previous knowledge distillation and fast sampling methods to achieve significant acceleration. Codes will be released in Github.

本研究针对扩散模型在生成图像时面临的重大计算开销问题，提出了一种新颖的一对多知识蒸馏（O2MKD）方法。这一方法通过将单个教师扩散模型蒸馏为多个学生扩散模型，使每个学生模型专注于不同的连续时间步骤，从而显著加快生成速度，实验结果显示在多个数据集上均实现了加速效果。

通过一对多知识蒸馏加速扩散模型