Denoising diffusion models have shown remarkable potential in various
generation tasks. The open-source large-scale text-to-image model, Stable
Diffusion, becomes prevalent as it can generate realistic artistic or facial
images with personalization through fine-tuning on a limited number of new
samples. However, this has raised privacy concerns as adversaries can acquire
facial images online and fine-tune text-to-image models for malicious editing,
leading to baseless scandals, defamation, and disruption to victims' lives.
Prior research efforts have focused on deriving adversarial loss from
conventional training processes for facial privacy protection through
adversarial perturbations. However, existing algorithms face two issues: 1)
they neglect the image-text fusion module, which is the vital module of
text-to-image diffusion models, and 2) their defensive performance is unstable
against different attacker prompts. In this paper, we propose the Adversarial
Decoupling Augmentation Framework (ADAF), addressing these issues by targeting
the image-text fusion module to enhance the defensive performance of facial
privacy protection algorithms. ADAF introduces multi-level text-related
augmentations for defense stability against various attacker prompts.
Concretely, considering the vision, text, and common unit space, we propose
Vision-Adversarial Loss, Prompt-Robust Augmentation, and Attention-Decoupling
Loss. Extensive experiments on CelebA-HQ and VGGFace2 demonstrate ADAF's
promising performance, surpassing existing algorithms.

本研究提出了一种基于 Adversarial Decoupling Augmentation Framework (ADAF) 的面部隐私保护算法，通过针对图像 - 文本融合模块增强面部隐私保护算法的防御性能，采用多级文本相关增强措施提高对各种攻击者提示的防御稳定性，并在 CelebA-HQ 和 VGGFace2 上进行广泛的实验，证明了 ADAF 比现有算法具有更好的性能。