Image style transfer aims to imbue digital imagery with the distinctive
attributes of style targets, such as colors, brushstrokes, shapes, whilst
concurrently preserving the semantic integrity of the content. Despite the
advancements in arbitrary style transfer methods, a prevalent challenge remains
the delicate equilibrium between content semantics and style attributes. Recent
developments in large-scale text-to-image diffusion models have heralded
unprecedented synthesis capabilities, albeit at the expense of relying on
extensive and often imprecise textual descriptions to delineate artistic
styles. Addressing these limitations, this paper introduces DiffStyler, a novel
approach that facilitates efficient and precise arbitrary image style transfer.
DiffStyler lies the utilization of a text-to-image Stable Diffusion model-based
LoRA to encapsulate the essence of style targets. This approach, coupled with
strategic cross-LoRA feature and attention injection, guides the style transfer
process. The foundation of our methodology is rooted in the observation that
LoRA maintains the spatial feature consistency of UNet, a discovery that
further inspired the development of a mask-wise style transfer technique. This
technique employs masks extracted through a pre-trained FastSAM model,
utilizing mask prompts to facilitate feature fusion during the denoising
process, thereby enabling localized style transfer that preserves the original
image's unaffected regions. Moreover, our approach accommodates multiple style
targets through the use of corresponding masks. Through extensive
experimentation, we demonstrate that DiffStyler surpasses previous methods in
achieving a more harmonious balance between content preservation and style
integration.

DiffStyler 是一种新方法，通过利用基于 LoRA 的文本到图像扩散模型来实现高效且精确的任意图像风格转移，该方法结合了跨 LoRA 特征和注意注入的策略，旨在在内容保持和风格整合之间取得更和谐的平衡。

DiffStyler：基于扩散的局部图像风格转移

DiffStyler: Diffusion-based Localized Image Style Transfer

In recent years, arbitrary image style transfer has attracted more and more
attention. Given a pair of content and style images, a stylized one is hoped
that retains the content from the former while catching style patterns from the
latter. However, it is difficult to simultaneously keep well the trade-off
between the content details and the style features. To stylize the image with
sufficient style patterns, the content details may be damaged and sometimes the
objects of images can not be distinguished clearly. For this reason, we present
a new transformer-based method named STT for image style transfer and an edge
loss which can enhance the content details apparently to avoid generating
blurred results for excessive rendering on style features. Qualitative and
quantitative experiments demonstrate that STT achieves comparable performance
to state-of-the-art image style transfer methods while alleviating the content
leak problem.

本研究介绍了一种新的基于转换器（transformer-based）方法和边缘损失（edge loss）的图像风格转换方法 STT，可以在保持较好的风格特征同时提升内容细节，避免产生过度渲染的模糊结果，并解决常见的内容泄漏问题。实验结果表明，STT 在图像风格转换方面具有与最先进的方法相当的性能。

基于 Transformer 的边缘增强图像风格迁移

Edge Enhanced Image Style Transfer via Transformers

In this work, we tackle the challenging problem of arbitrary image style
transfer using a novel style feature representation learning method. A suitable
style representation, as a key component in image stylization tasks, is
essential to achieve satisfactory results. Existing deep neural network based
approaches achieve reasonable results with the guidance from second-order
statistics such as Gram matrix of content features. However, they do not
leverage sufficient style information, which results in artifacts such as local
distortions and style inconsistency. To address these issues, we propose to
learn style representation directly from image features instead of their
second-order statistics, by analyzing the similarities and differences between
multiple styles and considering the style distribution. Specifically, we
present Contrastive Arbitrary Style Transfer (CAST), which is a new style
representation learning and style transfer method via contrastive learning. Our
framework consists of three key components, i.e., a multi-layer style projector
for style code encoding, a domain enhancement module for effective learning of
style distribution, and a generative network for image style transfer. We
conduct qualitative and quantitative evaluations comprehensively to demonstrate
that our approach achieves significantly better results compared to those
obtained via state-of-the-art methods. Code and models are available at
this https URL

本文提出一种使用深度对比学习的方法，通过学习图像特征来获得图像的样式表示，并使用生成网络进行样式转移。研究表明，相对于现有的基于二阶统计量的方法，这种方法能够更加有效地获取图像的样式信息并生成更好的结果。