In this work, we address two limitations of existing conditional diffusion
models: their slow inference speed due to the iterative denoising process and
their reliance on paired data for model fine-tuning. To tackle these issues, we
introduce a general method for adapting a single-step diffusion model to new
tasks and domains through adversarial learning objectives. Specifically, we
consolidate various modules of the vanilla latent diffusion model into a single
end-to-end generator network with small trainable weights, enhancing its
ability to preserve the input image structure while reducing overfitting. We
demonstrate that, for unpaired settings, our model CycleGAN-Turbo outperforms
existing GAN-based and diffusion-based methods for various scene translation
tasks, such as day-to-night conversion and adding/removing weather effects like
fog, snow, and rain. We extend our method to paired settings, where our model
pix2pix-Turbo is on par with recent works like Control-Net for Sketch2Photo and
Edge2Image, but with a single-step inference. This work suggests that
single-step diffusion models can serve as strong backbones for a range of GAN
learning objectives. Our code and models are available at
this https URL

通过对抗学习目标，将单步扩散模型通过整合各种模块成为单一的端到端生成网络，以提高其保留输入图像结构并减少过拟合的能力，我们的模型在非配对设置下优于现有的基于 GAN 和扩散的方法，实现了各种场景转换任务，如昼夜转换和添加 / 删除雾、雪和雨等天气效果；在配对设置中，我们的模型与现有的作品如 Sketch2Photo 和 Edge2Image 的 Control-Net 不相上下，但具有单步推理。