Drawing upon StyleGAN's expressivity and disentangled latent space, existing
2D approaches employ textual prompting to edit facial images with different
attributes. In contrast, 3D-aware approaches that generate faces at different
target poses require attribute-specific classifiers, learning separate model
weights for each attribute, and are not scalable for novel attributes. In this
work, we propose an efficient, plug-and-play, 3D-aware face editing framework
based on attribute-specific prompt learning, enabling the generation of facial
images with controllable attributes across various target poses. To this end,
we introduce a text-driven learnable style token-based latent attribute editor
(LAE). The LAE harnesses a pre-trained vision-language model to find
text-guided attribute-specific editing direction in the latent space of any
pre-trained 3D-aware GAN. It utilizes learnable style tokens and style mappers
to learn and transform this editing direction to 3D latent space. To train LAE
with multiple attributes, we use directional contrastive loss and style token
loss. Furthermore, to ensure view consistency and identity preservation across
different poses and attributes, we employ several 3D-aware identity and pose
preservation losses. Our experiments show that our proposed framework generates
high-quality images with 3D awareness and view consistency while maintaining
attribute-specific features. We demonstrate the effectiveness of our method on
different facial attributes, including hair color and style, expression, and
others. Code:
this https URL

提出了一种基于属性特定的提示学习的高效、即插即用的三维感知人脸编辑框架，该框架利用可学习的样式令牌和样式映射器，通过预先训练的视觉语言模型在任何预先训练的三维感知生成对抗网络的潜在空间中找到以文本为导向的属性特定编辑方向，并将其转化为三维潜在空间。通过多个三维感知的身份和姿势保持损失，保证了视角一致性和身份保持。实验证明，该方法能够生成具有三维感知和视角一致性的高质量图像，同时保持属性特定的特征。

通过属性特定提示学习实现高效的三维感知人脸图像编辑

Efficient 3D-Aware Facial Image Editing via Attribute-Specific Prompt  Learning

Recently, 3D-aware face editing has witnessed remarkable progress. Although
current approaches successfully perform mask-guided or text-based editing,
these properties have not been combined into a single method. To address this
limitation, we propose \textbf{MaTe3D}: mask-guided text-based 3D-aware
portrait editing. First, we propose a new SDF-based 3D generator. To better
perform masked-based editing (mainly happening in local areas), we propose SDF
and density consistency losses, aiming to effectively model both the global and
local representations jointly. Second, we introduce an inference-optimized
method. We introduce two techniques based on the SDS (Score Distillation
Sampling), including a blending SDS and a conditional SDS. The former aims to
overcome the mismatch problem between geometry and appearance, ultimately
harming fidelity. The conditional SDS contributes to further producing
satisfactory and stable results. Additionally, we create CatMask-HQ dataset, a
large-scale high-resolution cat face annotations. We perform experiments on
both the FFHQ and CatMask-HQ datasets to demonstrate the effectiveness of the
proposed method. Our method generates faithfully a edited 3D-aware face image
given a modified mask and a text prompt. Our code and models will be publicly
released.

提出了名为 MaTe3D 的基于蒙版导向的文本驱动的三维感知肖像编辑方法。通过引入一种新的基于 SDF 的三维生成器以及基于 SDS 的推理优化方法，有效地模拟了全局和局部特征，并针对性地处理了几何和外貌不匹配的问题，从而成功生成了编辑后的三维感知人脸图像。