Given an artistic portrait, recovering the latent photorealistic face that
preserves the subject's identity is challenging because the facial details are
often distorted or fully lost in artistic portraits. We develop an
Identity-preserving Face Recovery from Portraits (IFRP) method that utilizes a
Style Removal network (SRN) and a Discriminative Network (DN). Our SRN,
composed of an autoencoder with residual block-embedded skip connections, is
designed to transfer feature maps of stylized images to the feature maps of the
corresponding photorealistic faces. Owing to the Spatial Transformer Network
(STN), SRN automatically compensates for misalignments of stylized portraits to
output aligned realistic face images. To ensure the identity preservation, we
promote the recovered and ground truth faces to share similar visual features
via a distance measure which compares features of recovered and ground truth
faces extracted from a pre-trained FaceNet network. DN has multiple
convolutional and fully-connected layers, and its role is to enforce recovered
faces to be similar to authentic faces. Thus, we can recover high-quality
photorealistic faces from unaligned portraits while preserving the identity of
the face in an image. By conducting extensive evaluations on a large-scale
synthesized dataset and a hand-drawn sketch dataset, we demonstrate that our
method achieves superior face recovery and attains state-of-the-art results. In
addition, our method can recover photorealistic faces from unseen stylized
portraits, artistic paintings, and hand-drawn sketches.

本文提出了一种 IFRP 方法，结合了 SRN 和 DN 网络，通过 STN 自动矫正艺术肖像图像的定位问题，并使用 FaceNet 网络检测提取人脸特征，从而实现从不对齐的肖像中恢复出保留主体身份的高质量真实面部图像。