Despite recent advances in face recognition using deep learning, severe
accuracy drops are observed for large pose variations in unconstrained
environments. Learning pose-invariant features is one solution, but needs
expensively labeled large-scale data and carefully designed feature learning
algorithms. In this work, we focus on frontalizing faces in the wild under
various head poses, including extreme profile views. We propose a novel deep 3D
Morphable Model (3DMM) conditioned Face Frontalization Generative Adversarial
Network (GAN), termed as FF-GAN, to generate neutral head pose face images. Our
framework differs from both traditional GANs and 3DMM based modeling.
Incorporating 3DMM into the GAN structure provides shape and appearance priors
for fast convergence with less training data, while also supporting end-to-end
training. The 3DMM-conditioned GAN employs not only the discriminator and
generator loss but also a new masked symmetry loss to retain visual quality
under occlusions, besides an identity loss to recover high frequency
information. Experiments on face recognition, landmark localization and 3D
reconstruction consistently show the advantage of our frontalization method on
faces in the wild datasets.

本文提出了一种利用深度学习中的 3D 可塑模型进行面部正面化的生成式对抗网络，可以有效提高识别率和处理不同姿势的面部图像。