Diffusion models have revolutionized text-to-image generation, but their real-world applications are hampered by the extensive time needed for hundreds of diffusion steps. Although progressive distillation has been proposed to speed up diffusion sampling to 2-8 steps, it still falls short in one-step generation, and necessitates training multiple student models, which is highly parameter-extensive and time-consuming. To overcome these limitations, we introduce High-frequency-Promoting Adaptation (HiPA), a parameter-efficient approach to enable one-step text-to-image diffusion. Grounded in the insight that high-frequency information is essential but highly lacking in one-step diffusion, HiPA focuses on training one-step, low-rank adaptors to specifically enhance the under-represented high-frequency abilities of advanced diffusion models. The learned adaptors empower these diffusion models to generate high-quality images in just a single step. Compared with progressive distillation, HiPA achieves much better performance in one-step text-to-image generation (37.3 $\rightarrow$ 23.8 in FID-5k on MS-COCO 2017) and 28.6x training speed-up (108.8 $\rightarrow$ 3.8 A100 GPU days), requiring only 0.04% training parameters (7,740 million $\rightarrow$ 3.3 million). We also demonstrate HiPA's effectiveness in text-guided image editing, inpainting and super-resolution tasks, where our adapted models consistently deliver high-quality outputs in just one diffusion step. The source code will be released.

高频率促进适应（HiPA）是一种参数高效的方法，通过训练一步、低秩适配器来增强先进扩散模型中高频率能力的不足，使其能够在单一步骤中生成高质量图像。与渐进蒸馏相比，HiPA在一步文本到图像生成中具有更好的性能（FID-5k在MS-COCO 2017上从37.3降至23.8），并具有28.6倍的训练加速（108.8到3.8 A100 GPU天），仅需要0.04％的训练参数（77.4亿降至330万）。

HiPA: 通过高频率推广适应实现一步式文本到图像扩散模型