Style transfer aims to fuse the artistic representation of a style image with the structural information of a content image. Existing methods train specific networks or utilize pre-trained models to learn content and style features. However, they rely solely on textual or spatial representations that are inadequate to achieve the balance between content and style. In this work, we propose a novel and training-free approach for style transfer, combining textual embedding with spatial features and separating the injection of content or style. Specifically, we adopt the BLIP-2 encoder to extract the textual representation of the style image. We utilize the DDIM inversion technique to extract intermediate embeddings in content and style branches as spatial features. Finally, we harness the step-by-step property of diffusion models by separating the injection of content and style in the target branch, which improves the balance between content preservation and style fusion. Various experiments have demonstrated the effectiveness and robustness of our proposed DiffeseST for achieving balanced and controllable style transfer results, as well as the potential to extend to other tasks.

本研究解决了现有风格迁移方法在内容和风格之间平衡不足的问题。我们提出了一种新颖的无训练方法，通过将文本嵌入与空间特征相结合，并分开注入内容和风格，从而改善风格迁移效果。实验表明，该方法在实现平衡和可控的风格迁移结果方面表现出色，并具有扩展到其他任务的潜力。

DiffuseST：释放扩散模型在风格迁移中的能力