Current makeup transfer methods are limited to simple makeup styles, making
them difficult to apply in real-world scenarios. In this paper, we introduce
Stable-Makeup, a novel diffusion-based makeup transfer method capable of
robustly transferring a wide range of real-world makeup, onto user-provided
faces. Stable-Makeup is based on a pre-trained diffusion model and utilizes a
Detail-Preserving (D-P) makeup encoder to encode makeup details. It also
employs content and structural control modules to preserve the content and
structural information of the source image. With the aid of our newly added
makeup cross-attention layers in U-Net, we can accurately transfer the detailed
makeup to the corresponding position in the source image. After
content-structure decoupling training, Stable-Makeup can maintain content and
the facial structure of the source image. Moreover, our method has demonstrated
strong robustness and generalizability, making it applicable to varioustasks
such as cross-domain makeup transfer, makeup-guided text-to-image generation
and so on. Extensive experiments have demonstrated that our approach delivers
state-of-the-art (SOTA) results among existing makeup transfer methods and
exhibits a highly promising with broad potential applications in various
related fields.

我们介绍了一种名为 Stable-Makeup 的新型基于扩散的化妆迁移方法，能够鲁棒地将各种真实世界的化妆应用到用户提供的人脸上。该方法通过预训练的扩散模型和 Detail-Preserving（D-P）化妆编码器来编码化妆细节，并利用内容和结构控制模块来保留源图像的内容和结构信息。我们的方法还通过在 U-Net 中添加化妆交叉注意力层，可以准确地将细节化妆转移到源图像的相应位置。Stable-Makeup 通过内容结构解耦训练可以保持源图像的内容和面部结构。此外，我们的方法表现出较强的鲁棒性和泛化能力，可应用于交叉领域的化妆迁移、以化妆为导向的文本到图像生成等多个任务。大量实验表明，我们的方法在现有化妆迁移方法中取得了最先进的结果，并在多个相关领域具有广泛的潜在应用。