We propose a method to distill a complex multistep diffusion model into a
single-step conditional GAN student model, dramatically accelerating inference,
while preserving image quality. Our approach interprets diffusion distillation
as a paired image-to-image translation task, using noise-to-image pairs of the
diffusion model's ODE trajectory. For efficient regression loss computation, we
propose E-LatentLPIPS, a perceptual loss operating directly in diffusion
model's latent space, utilizing an ensemble of augmentations. Furthermore, we
adapt a diffusion model to construct a multi-scale discriminator with a text
alignment loss to build an effective conditional GAN-based formulation.
E-LatentLPIPS converges more efficiently than many existing distillation
methods, even accounting for dataset construction costs. We demonstrate that
our one-step generator outperforms cutting-edge one-step diffusion distillation
models - DMD, SDXL-Turbo, and SDXL-Lightning - on the zero-shot COCO benchmark.

我们提出了一种方法将复杂的多步扩散模型提炼为单步有条件生成对抗网络学生模型，从而大大加速推理过程，同时保持图像质量。我们的方法将扩散提炼解释为一种对应的图像到图像转换任务，使用扩散模型 ODE 轨迹的噪声到图像对。为了进行高效的回归损失计算，我们提出了一种在扩散模型的潜空间中直接操作的感知损失 E-LatentLPIPS，利用增强的集合。此外，我们改进了扩散模型，构建了一个多尺度鉴别器，具有文本对齐损失，建立了一种有效的基于条件生成对抗网络的公式。即使考虑到数据集构建成本，E-LatentLPIPS 也比许多现有的提炼方法更高效。我们证明我们的一步生成器在零样本 COCO 基准测试上优于最先进的一步扩散提炼模型 - DMD、SDXL-Turbo 和 SDXL-Lightning。

将扩散模型融合为条件生成对抗网络

Distilling Diffusion Models into Conditional GANs

We present a learned image compression system based on GANs, operating at
extremely low bitrates. Our proposed framework combines an encoder,
decoder/generator and a multi-scale discriminator, which we train jointly for a
generative learned compression objective. The model synthesizes details it
cannot afford to store, obtaining visually pleasing results at bitrates where
previous methods fail and show strong artifacts. Furthermore, if a semantic
label map of the original image is available, our method can fully synthesize
unimportant regions in the decoded image such as streets and trees from the
label map, proportionally reducing the storage cost. A user study confirms that
for low bitrates, our approach is preferred to state-of-the-art methods, even
when they use more than double the bits.

本研究基于生成对抗网络构建了一个图像压缩系统，其中包括编码器、解码器 / 生成器和多尺度鉴别器，并使用全生成式学习压缩目标。模型可合成存储受限的细节，实现在比之前方法失败且出现严重伪影的比特率下，较视觉上令人满意的结果。此外，如果有原始图像的语义标签映射可用，则本方法可以从标签映射中合成出解码后图像的不重要区域，例如街道和树，并相应地减少存储成本。一个用户研究证实，即使使用两倍以上的比特，低比特率下我们的方法都优于现有技术。