In this work, we focus on exploring explicit fine-grained control of
generative facial image editing, all while generating faithful and consistent
personalized facial appearances. We identify the key challenge of this task as
the exploration of disentangled conditional control in the generation process,
and accordingly propose a novel diffusion-based framework, named
DisControlFace, comprising two decoupled components. Specifically, we leverage
an off-the-shelf diffusion reconstruction model as the backbone and freeze its
pre-trained weights, which helps to reduce identity shift and recover
editing-unrelated details of the input image. Furthermore, we construct a
parallel control network that is compatible with the reconstruction backbone to
generate spatial control conditions based on estimated explicit face
parameters. Finally, we further reformulate the training pipeline into a
masked-autoencoding form to effectively achieve disentangled training of our
DisControlFace. Our DisControlNet can perform robust editing on any facial
image through training on large-scale 2D in-the-wild portraits and also
supports low-cost fine-tuning with few additional images to further learn
diverse personalized priors of a specific person. Extensive experiments
demonstrate that DisControlFace can generate realistic facial images
corresponding to various face control conditions, while significantly improving
the preservation of the personalized facial details.

我们的研究侧重于探索生成性面部图像编辑中明确的细粒度控制，同时生成忠实和一致的个性化面部外貌。我们确定了这项任务的关键挑战是在生成过程中对解缠条件控制的探索，并相应地提出了一种新的基于扩散的框架，称为 DisControlFace，包括两个解耦的组件。