Image Style Transfer (IST) is an interdisciplinary topic of computer vision
and art that continuously attracts researchers' interests. Different from
traditional Image-guided Image Style Transfer (IIST) methods that require a
style reference image as input to define the desired style, recent works start
to tackle the problem in a text-guided manner, i.e., Text-guided Image Style
Transfer (TIST). Compared to IIST, such approaches provide more flexibility
with text-specified styles, which are useful in scenarios where the style is
hard to define with reference images. Unfortunately, many TIST approaches
produce undesirable artifacts in the transferred images. To address this issue,
we present a novel method to achieve much improved style transfer based on text
guidance. Meanwhile, to offer more flexibility than IIST and TIST, our method
allows style inputs from multiple sources and modalities, enabling
MultiModality-guided Image Style Transfer (MMIST). Specifically, we realize
MMIST with a novel cross-modal GAN inversion method, which generates style
representations consistent with specified styles. Such style representations
facilitate style transfer and in principle generalize any IIST methods to
MMIST. Large-scale experiments and user studies demonstrate that our method
achieves state-of-the-art performance on TIST task. Furthermore, comprehensive
qualitative results confirm the effectiveness of our method on MMIST task and
cross-modal style interpolation.

基于文本引导的多模态图像风格迁移方法，使用了 GAN 反演技术生成与指定风格一致的风格表示，实现了在图像风格迁移任务中最先进的性能，并通过全面的定性结果验证了在多模态任务和跨模态风格插值任务中的有效性。

多模态引导的图像风格转移：跨模态生成对抗网络反演

Multimodality-guided Image Style Transfer using Cross-modal GAN  Inversion

Diffusion models have shown great promise in text-guided image style
transfer, but there is a trade-off between style transformation and content
preservation due to their stochastic nature. Existing methods require
computationally expensive fine-tuning of diffusion models or additional neural
network. To address this, here we propose a zero-shot contrastive loss for
diffusion models that doesn't require additional fine-tuning or auxiliary
networks. By leveraging patch-wise contrastive loss between generated samples
and original image embeddings in the pre-trained diffusion model, our method
can generate images with the same semantic content as the source image in a
zero-shot manner. Our approach outperforms existing methods while preserving
content and requiring no additional training, not only for image style transfer
but also for image-to-image translation and manipulation. Our experimental
results validate the effectiveness of our proposed method.

本文提出了一种零样式迁移的扩散模型，该零样式迁移使用零额外训练或辅助网络的对比损失来处理内容保留问题，并在图像样式转换和图像到图像翻译中取得了优于现有方法的效果。