Face stylization refers to the transformation of a face into a specific
portrait style. However, current methods require the use of example-based
adaptation approaches to fine-tune pre-trained generative models so that they
demand lots of time and storage space and fail to achieve detailed style
transformation. This paper proposes a training-free face stylization framework,
named Portrait Diffusion. This framework leverages off-the-shelf text-to-image
diffusion models, eliminating the need for fine-tuning specific examples.
Specifically, the content and style images are first inverted into latent
codes. Then, during image reconstruction using the corresponding latent code,
the content and style features in the attention space are delicately blended
through a modified self-attention operation called Style Attention Control.
Additionally, a Chain-of-Painting method is proposed for the gradual redrawing
of unsatisfactory areas from rough adjustments to fine-tuning. Extensive
experiments validate the effectiveness of our Portrait Diffusion method and
demonstrate the superiority of Chain-of-Painting in achieving precise face
stylization. Code will be released at
https://github.com/liujin112/PortraitDiffusion.

本研究提出了一种无需训练的人像风格化框架，名为 Portrait Diffusion，通过使用现成的文本到图像扩散模型，在注意力空间中通过样式关注控制精细地融合内容和样式特征，并提出了一种渐进涂画方法 Chain-of-Painting，从粗糙调整到精细调优，有效实现了面部风格化。