Diffusion distillation represents a highly promising direction for achieving
faithful text-to-image generation in a few sampling steps. However, despite
recent successes, existing distilled models still do not provide the full
spectrum of diffusion abilities, such as real image inversion, which enables
many precise image manipulation methods. This work aims to enrich distilled
text-to-image diffusion models with the ability to effectively encode real
images into their latent space. To this end, we introduce invertible
Consistency Distillation (iCD), a generalized consistency distillation
framework that facilitates both high-quality image synthesis and accurate image
encoding in only 3-4 inference steps. Though the inversion problem for
text-to-image diffusion models gets exacerbated by high classifier-free
guidance scales, we notice that dynamic guidance significantly reduces
reconstruction errors without noticeable degradation in generation performance.
As a result, we demonstrate that iCD equipped with dynamic guidance may serve
as a highly effective tool for zero-shot text-guided image editing, competing
with more expensive state-of-the-art alternatives.

通过应用可逆一致性蒸馏 (invertible Consistency Distillation, iCD) 框架，实现了在少于 4 个推理步骤中高质量图像合成和准确图像编码的目标，使得具有动态引导的 iCD 成为零样本文本引导图像编辑的高效工具。

用于文本引导图像编辑的可逆一致性蒸馏方法（约 7 步）

Invertible Consistency Distillation for Text-Guided Image Editing in  Around 7 Steps

We propose a method to distill a complex multistep diffusion model into a
single-step conditional GAN student model, dramatically accelerating inference,
while preserving image quality. Our approach interprets diffusion distillation
as a paired image-to-image translation task, using noise-to-image pairs of the
diffusion model's ODE trajectory. For efficient regression loss computation, we
propose E-LatentLPIPS, a perceptual loss operating directly in diffusion
model's latent space, utilizing an ensemble of augmentations. Furthermore, we
adapt a diffusion model to construct a multi-scale discriminator with a text
alignment loss to build an effective conditional GAN-based formulation.
E-LatentLPIPS converges more efficiently than many existing distillation
methods, even accounting for dataset construction costs. We demonstrate that
our one-step generator outperforms cutting-edge one-step diffusion distillation
models - DMD, SDXL-Turbo, and SDXL-Lightning - on the zero-shot COCO benchmark.

我们提出了一种方法将复杂的多步扩散模型提炼为单步有条件生成对抗网络学生模型，从而大大加速推理过程，同时保持图像质量。我们的方法将扩散提炼解释为一种对应的图像到图像转换任务，使用扩散模型 ODE 轨迹的噪声到图像对。为了进行高效的回归损失计算，我们提出了一种在扩散模型的潜空间中直接操作的感知损失 E-LatentLPIPS，利用增强的集合。此外，我们改进了扩散模型，构建了一个多尺度鉴别器，具有文本对齐损失，建立了一种有效的基于条件生成对抗网络的公式。即使考虑到数据集构建成本，E-LatentLPIPS 也比许多现有的提炼方法更高效。我们证明我们的一步生成器在零样本 COCO 基准测试上优于最先进的一步扩散提炼模型 - DMD、SDXL-Turbo 和 SDXL-Lightning。

将扩散模型融合为条件生成对抗网络

Distilling Diffusion Models into Conditional GANs

We propose a diffusion distillation method that achieves new state-of-the-art
in one-step/few-step 1024px text-to-image generation based on SDXL. Our method
combines progressive and adversarial distillation to achieve a balance between
quality and mode coverage. In this paper, we discuss the theoretical analysis,
discriminator design, model formulation, and training techniques. We
open-source our distilled SDXL-Lightning models both as LoRA and full UNet
weights.

我们提出了一种扩散蒸馏方法，基于 SDXL，在一步 / 几步 1024 像素的文本到图像生成任务上达到了新的最先进水平。我们的方法结合了渐进式和对抗性蒸馏来在质量和模态覆盖之间找到平衡。在本文中，我们讨论了理论分析、判别器设计、模型公式和训练技术。我们以 LoRA 和完整的 UNet 权重形式开源了我们蒸馏的 SDXL-Lightning 模型。