Latent Consistency Models (LCMs) have achieved impressive performance in
accelerating text-to-image generative tasks, producing high-quality images with
minimal inference steps. LCMs are distilled from pre-trained latent diffusion
models (LDMs), requiring only ~32 A100 GPU training hours. This report further
extends LCMs' potential in two aspects: First, by applying LoRA distillation to
Stable-Diffusion models including SD-V1.5, SSD-1B, and SDXL, we have expanded
LCM's scope to larger models with significantly less memory consumption,
achieving superior image generation quality. Second, we identify the LoRA
parameters obtained through LCM distillation as a universal Stable-Diffusion
acceleration module, named LCM-LoRA. LCM-LoRA can be directly plugged into
various Stable-Diffusion fine-tuned models or LoRAs without training, thus
representing a universally applicable accelerator for diverse image generation
tasks. Compared with previous numerical PF-ODE solvers such as DDIM,
DPM-Solver, LCM-LoRA can be viewed as a plug-in neural PF-ODE solver that
possesses strong generalization abilities. Project page:
this https URL

通过将 LoRA 蒸馏应用于包括 SD-V1.5、SSD-1B 和 SDXL 的稳定扩散模型，扩展了潜在一致性模型 (LCMs) 的潜力，减少内存消耗，提高图像生成质量，并将通过 LCM 蒸馏得到的 LoRA 参数作为稳定扩散加速模块命名为 LCM-LoRA，该模块可以直接插入各种经过微调的稳定扩散模型或 LoRAs 中，从而成为多样的图像生成任务中可广泛应用的加速器。

LCM-LoRA：一种通用的稳定扩散加速模块

LCM-LoRA: A Universal Stable-Diffusion Acceleration Module

Latent Diffusion models (LDMs) have achieved remarkable results in
synthesizing high-resolution images. However, the iterative sampling process is
computationally intensive and leads to slow generation. Inspired by Consistency
Models (song et al.), we propose Latent Consistency Models (LCMs), enabling
swift inference with minimal steps on any pre-trained LDMs, including Stable
Diffusion (rombach et al). Viewing the guided reverse diffusion process as
solving an augmented probability flow ODE (PF-ODE), LCMs are designed to
directly predict the solution of such ODE in latent space, mitigating the need
for numerous iterations and allowing rapid, high-fidelity sampling. Efficiently
distilled from pre-trained classifier-free guided diffusion models, a
high-quality 768 x 768 2~4-step LCM takes only 32 A100 GPU hours for training.
Furthermore, we introduce Latent Consistency Fine-tuning (LCF), a novel method
that is tailored for fine-tuning LCMs on customized image datasets. Evaluation
on the LAION-5B-Aesthetics dataset demonstrates that LCMs achieve
state-of-the-art text-to-image generation performance with few-step inference.
Project Page: this https URL

利用潜在一致性模型（LCMs）解决迭代抽样过程在高分辨率图像合成中的计算负担和生成速度慢的问题，LCMs 通过直接预测导引反向扩散过程的 ODE 解决方案在潜空间中，实现了快速、高保真度的采样。