Facial Appearance Editing (FAE) aims to modify physical attributes, such as
pose, expression and lighting, of human facial images while preserving
attributes like identity and background, showing great importance in
photograph. In spite of the great progress in this area, current researches
generally meet three challenges: low generation fidelity, poor attribute
preservation, and inefficient inference. To overcome above challenges, this
paper presents DiffFAE, a one-stage and highly-efficient diffusion-based
framework tailored for high-fidelity FAE. For high-fidelity query attributes
transfer, we adopt Space-sensitive Physical Customization (SPC), which ensures
the fidelity and generalization ability by utilizing rendering texture derived
from 3D Morphable Model (3DMM). In order to preserve source attributes, we
introduce the Region-responsive Semantic Composition (RSC). This module is
guided to learn decoupled source-regarding features, thereby better preserving
the identity and alleviating artifacts from non-facial attributes such as hair,
clothes, and background. We further introduce a consistency regularization for
our pipeline to enhance editing controllability by leveraging prior knowledge
in the attention matrices of diffusion model. Extensive experiments demonstrate
the superiority of DiffFAE over existing methods, achieving state-of-the-art
performance in facial appearance editing.

DiffFAE 是一个基于扩散的高效一阶段框架，旨在实现高保真的面部外貌编辑。通过使用基于 3D Morphable Model（3DMM）的渲染纹理，采用空间敏感的物理定制（SPC）来确保高保真度和泛化能力；通过引入区域响应的语义组合（RSC）模块来保留源属性；通过一致性正则化来增强编辑可控性。实验证明，DiffFAE 在面部外貌编辑方面具有优越性能，达到了领先水平。