Face photo-sketch synthesis and recognition has many applications in digital
entertainment and law enforcement. Recently, generative adversarial networks
(GANs) based methods have significantly improved the quality of image
synthesis, but they have not explicitly considered the purpose of recognition.
In this paper, we first propose an Identity-Aware CycleGAN (IACycleGAN) model
that applies a new perceptual loss to supervise the image generation network.
It improves CycleGAN on photo-sketch synthesis by paying more attention to the
synthesis of key facial regions, such as eyes and nose, which are important for
identity recognition. Furthermore, we develop a mutual optimization procedure
between the synthesis model and the recognition model, which iteratively
synthesizes better images by IACycleGAN and enhances the recognition model by
the triplet loss of the generated and real samples. Extensive experiments are
performed on both photo-tosketch and sketch-to-photo tasks using the widely
used CUFS and CUFSF databases. The results show that the proposed method
performs better than several state-of-the-art methods in terms of both
synthetic image quality and photo-sketch recognition accuracy.

本研究提出了一种基于 IACycleGAN 模型的人脸照片 - 草图合成算法，采用感知损失监督图像生成网络以改进 CycleGAN，在关键的面部区域（如眼睛和鼻子）的合成过程中更加注重识别，同时还使用合成样本和真实样本的三元组损失增强了识别模型，在 CUFS 和 CUFSF 数据库上的实验结果表明，所提出的方法在合成图像质量和草图 - 照片识别精度方面均优于多个现有的先进方法。

用于人脸照片 - 素描合成和识别的身份感知 CycleGAN

Identity-Aware CycleGAN for Face Photo-Sketch Synthesis and Recognition

Face photo-sketch synthesis aims at generating a facial sketch/photo
conditioned on a given photo/sketch. It is of wide applications including
digital entertainment and law enforcement. Precisely depicting face
photos/sketches remains challenging due to the restrictions on structural
realism and textural consistency. While existing methods achieve compelling
results, they mostly yield blurred effects and great deformation over various
facial components, leading to the unrealistic feeling of synthesized images. To
tackle this challenge, in this work, we propose to use the facial composition
information to help the synthesis of face sketch/photo. Specially, we propose a
novel composition-aided generative adversarial network (CA-GAN) for face
photo-sketch synthesis. In CA-GAN, we utilize paired inputs including a face
photo/sketch and the corresponding pixel-wise face labels for generating a
sketch/photo. In addition, to focus training on hard-generated components and
delicate facial structures, we propose a compositional reconstruction loss.
Finally, we use stacked CA-GANs (SCA-GAN) to further rectify defects and add
compelling details. Experimental results show that our method is capable of
generating both visually comfortable and identity-preserving face
sketches/photos over a wide range of challenging data. Our method achieves the
state-of-the-art quality, reducing best previous Frechet Inception distance
(FID) by a large margin. Besides, we demonstrate that the proposed method is of
considerable generalization ability. We have made our code and results publicly
available: this https URL

本文提出一个基于面部构图信息的 Composition-aided Generative Adversarial Network (CA-GAN) 来生成面部素描 / 照片。通过使用成对的输入包括面部照片 / 素描和相应的面部标签来进行合成，最终使用堆叠的 CA-GANs (SCA-GAN) 去修正缺陷并增加有吸引力的细节，实验结果表明，我们的方法能够在广泛的具有挑战性的数据范围内生成视觉舒适、保持特征的面部素描 / 照片，并且取得了最佳的生成效果。