In this work, we introduce a new approach for artistic face stylization.
Despite existing methods achieving impressive results in this task, there is
still room for improvement in generating high-quality stylized faces with
diverse styles and accurate facial reconstruction. Our proposed framework,
MMFS, supports multi-modal face stylization by leveraging the strengths of
StyleGAN and integrates it into an encoder-decoder architecture. Specifically,
we use the mid-resolution and high-resolution layers of StyleGAN as the decoder
to generate high-quality faces, while aligning its low-resolution layer with
the encoder to extract and preserve input facial details. We also introduce a
two-stage training strategy, where we train the encoder in the first stage to
align the feature maps with StyleGAN and enable a faithful reconstruction of
input faces. In the second stage, the entire network is fine-tuned with
artistic data for stylized face generation. To enable the fine-tuned model to
be applied in zero-shot and one-shot stylization tasks, we train an additional
mapping network from the large-scale Contrastive-Language-Image-Pre-training
(CLIP) space to a latent $w+$ space of fine-tuned StyleGAN. Qualitative and
quantitative experiments show that our framework achieves superior face
stylization performance in both one-shot and zero-shot stylization tasks,
outperforming state-of-the-art methods by a large margin.

本文提出了一种支持多模式脸部艺术风格化的框架，利用 StyleGAN 的优势，并将其集成到编码器 - 解码器架构中进行高质量的面部生成，输出结果显示该框架在一次和零次风格化任务中均能实现比现有方法更出色的面部风格化性能。