Recent Customized Portrait Generation (CPG) methods, taking a facial image and a textual prompt as inputs, have attracted substantial attention. Although these methods generate high-fidelity portraits, they fail to prevent the generated portraits from being tracked and misused by malicious face recognition systems. To address this, this paper proposes a Customized Portrait Generation framework with facial Adversarial attacks (Adv-CPG). Specifically, to achieve facial privacy protection, we devise a lightweight local ID encryptor and an encryption enhancer. They implement progressive double-layer encryption protection by directly injecting the target identity and adding additional identity guidance, respectively. Furthermore, to accomplish fine-grained and personalized portrait generation, we develop a multi-modal image customizer capable of generating controlled fine-grained facial features. To the best of our knowledge, Adv-CPG is the first study that introduces facial adversarial attacks into CPG. Extensive experiments demonstrate the superiority of Adv-CPG, e.g., the average attack success rate of the proposed Adv-CPG is 28.1% and 2.86% higher compared to the SOTA noise-based attack methods and unconstrained attack methods, respectively.

本研究针对现有定制肖像生成方法在数据隐私方面的不足，提出了一种新的带有面部对抗攻击的定制肖像生成框架Adv-CPG。该框架引入了轻量级的本地ID加密器和加密增强器，实现了双层加密保护，且具备生成精细和个性化肖像的能力。实验结果表明，Adv-CPG在攻击成功率上显著优于现有方法，展示了其在肖像隐私保护领域的潜在影响。

带有面部对抗攻击的定制肖像生成框架 Adv-CPG