It is challenging to inpaint face images in the wild, due to the large
variation of appearance, such as different poses, expressions and occlusions. A
good inpainting algorithm should guarantee the realism of output, including the
topological structure among eyes, nose and mouth, as well as the attribute
consistency on pose, gender, ethnicity, expression, etc. This paper studies an
effective deep learning based strategy to deal with these issues, which
comprises of a facial landmark predicting subnet and an image inpainting
subnet. Concretely, given partial observation, the landmark predictor aims to
provide the structural information (e.g. topological relationship and
expression) of incomplete faces, while the inpaintor is to generate plausible
appearance (e.g. gender and ethnicity) conditioned on the predicted landmarks.
Experiments on the CelebA-HQ and CelebA datasets are conducted to reveal the
efficacy of our design and, to demonstrate its superiority over
state-of-the-art alternatives both qualitatively and quantitatively. In
addition, we assume that high-quality completed faces together with their
landmarks can be utilized as augmented data to further improve the performance
of (any) landmark predictor, which is corroborated by experimental results on
the 300W and WFLW datasets.

本文提出了一种基于深度学习的人脸修复算法，结合人脸关键点预测和图像修复，实现了对人脸姿态、表情和遮挡等多方面的修复。在 CelebA-HQ 和 CelebA 数据集上的实验表明，该算法在视觉效果和定量评估上均超过了现有算法。同时，本文还发现高质量的修复图像和其关键点可以有效提高关键点预测模型的性能，在 300W 和 WFLW 数据集上也得到了验证。