Face inpainting is important in various applications, such as photo
restoration, image editing, and virtual reality. Despite the significant
advances in face generative models, ensuring that a person's unique facial
identity is maintained during the inpainting process is still an elusive goal.
Current state-of-the-art techniques, exemplified by MyStyle, necessitate
resource-intensive fine-tuning and a substantial number of images for each new
identity. Furthermore, existing methods often fall short in accommodating
user-specified semantic attributes, such as beard or expression. To improve
inpainting results, and reduce the computational complexity during inference,
this paper proposes the use of Parallel Visual Attention (PVA) in conjunction
with diffusion models. Specifically, we insert parallel attention matrices to
each cross-attention module in the denoising network, which attends to features
extracted from reference images by an identity encoder. We train the added
attention modules and identity encoder on CelebAHQ-IDI, a dataset proposed for
identity-preserving face inpainting. Experiments demonstrate that PVA attains
unparalleled identity resemblance in both face inpainting and face inpainting
with language guidance tasks, in comparison to various benchmarks, including
MyStyle, Paint by Example, and Custom Diffusion. Our findings reveal that PVA
ensures good identity preservation while offering effective
language-controllability. Additionally, in contrast to Custom Diffusion, PVA
requires just 40 fine-tuning steps for each new identity, which translates to a
significant speed increase of over 20 times.

本文提出了使用并行视觉注意力（Parallel Visual Attention，PVA）和扩散模型相结合的方法，实现面部修复过程中的身份保留，达到了无与伦比的身份相似性，并具备有效的语言可控性。与各种基准方法相比，PVA 在面部修复和具有语言指导任务的面部修复中都表现出优异的结果。与 Custom Diffusion 相比，PVA 仅需每个新身份进行 40 次微调步骤，大大提高了速度，超过 20 倍。