Recent developments in face restoration have achieved remarkable results in
producing high-quality and lifelike outputs. The stunning results however often
fail to be faithful with respect to the identity of the person as the models
lack necessary context. In this paper, we explore the potential of personalized
face restoration with diffusion models. In our approach a restoration model is
personalized using a few images of the identity, leading to tailored
restoration with respect to the identity while retaining fine-grained details.
By using independent trainable blocks for personalization, the rich prior of a
base restoration model can be exploited to its fullest. To avoid the model
relying on parts of identity left in the conditioning low-quality images, a
generative regularizer is employed. With a learnable parameter, the model
learns to balance between the details generated based on the input image and
the degree of personalization. Moreover, we improve the training pipeline of
face restoration models to enable an alignment-free approach. We showcase the
robust capabilities of our approach in several real-world scenarios with
multiple identities, demonstrating our method's ability to generate
fine-grained details with faithful restoration. In the user study we evaluate
the perceptual quality and faithfulness of the genereated details, with our
method being voted best 61% of the time compared to the second best with 25% of
the votes.

我们探索了个性化人脸修复模型的潜力，通过使用扩散模型个性化恢复模型，保留细节的同时实现对身份的量身定制，通过使用独立的可训练模块以充分利用基础恢复模型的丰富先验信息，并通过引入生成规则器来避免模型依赖低质量图像中遗留的身份部分，我们展示了我们的方法在多个真实场景中对多个身份的鲁棒性能，并通过用户研究评估了生成细节的感知质量和忠实度，我们的方法被评为最佳的 61%。