The rapid development of generative diffusion models has significantly
advanced the field of style transfer. However, most current style transfer
methods based on diffusion models typically involve a slow iterative
optimization process, e.g., model fine-tuning and textual inversion of style
concept. In this paper, we introduce FreeStyle, an innovative style transfer
method built upon a pre-trained large diffusion model, requiring no further
optimization. Besides, our method enables style transfer only through a text
description of the desired style, eliminating the necessity of style images.
Specifically, we propose a dual-stream encoder and single-stream decoder
architecture, replacing the conventional U-Net in diffusion models. In the
dual-stream encoder, two distinct branches take the content image and style
text prompt as inputs, achieving content and style decoupling. In the decoder,
we further modulate features from the dual streams based on a given content
image and the corresponding style text prompt for precise style transfer. Our
experimental results demonstrate high-quality synthesis and fidelity of our
method across various content images and style text prompts. The code and more
results are available at our project
website:this https URL

本文介绍了一种创新的基于预训练大型扩散模型的风格转换方法 FreeStyle，无需进一步优化，通过对所需风格的文本描述，实现风格转换，并且实验结果表明了其在不同内容图像和风格文本提示下的高质量综合和保真度。