Blind face restoration is a highly ill-posed problem due to the lack of necessary context. Although existing methods produce high-quality outputs, they often fail to faithfully preserve the individual's identity. In this paper, we propose a personalized face restoration method, FaceMe, based on a diffusion model. Given a single or a few reference images, we use an identity encoder to extract identity-related features, which serve as prompts to guide the diffusion model in restoring high-quality and identity-consistent facial images. By simply combining identity-related features, we effectively minimize the impact of identity-irrelevant features during training and support any number of reference image inputs during inference. Additionally, thanks to the robustness of the identity encoder, synthesized images can be used as reference images during training, and identity changing during inference does not require fine-tuning the model. We also propose a pipeline for constructing a reference image training pool that simulates the poses and expressions that may appear in real-world scenarios. Experimental results demonstrate that our FaceMe can restore high-quality facial images while maintaining identity consistency, achieving excellent performance and robustness.

本文解决了盲人脸修复中由于缺乏必要上下文而导致的身份保持问题。我们提出了一种基于扩散模型的个性化人脸修复方法FaceMe，通过提取身份相关特征，引导模型在恢复高质量人脸图像的同时保持身份一致性。实验表明，FaceMe在图像恢复中表现出色，具有很好的鲁棒性和应用潜力。

FaceMe：鲁棒盲人脸修复与个人身份识别