Face photo-sketch synthesis and recognition has many applications in digital
entertainment and law enforcement. Recently, generative adversarial networks
(GANs) based methods have significantly improved the quality of image
synthesis, but they have not explicitly considered the purpose of recognition.
In this paper, we first propose an Identity-Aware CycleGAN (IACycleGAN) model
that applies a new perceptual loss to supervise the image generation network.
It improves CycleGAN on photo-sketch synthesis by paying more attention to the
synthesis of key facial regions, such as eyes and nose, which are important for
identity recognition. Furthermore, we develop a mutual optimization procedure
between the synthesis model and the recognition model, which iteratively
synthesizes better images by IACycleGAN and enhances the recognition model by
the triplet loss of the generated and real samples. Extensive experiments are
performed on both photo-tosketch and sketch-to-photo tasks using the widely
used CUFS and CUFSF databases. The results show that the proposed method
performs better than several state-of-the-art methods in terms of both
synthetic image quality and photo-sketch recognition accuracy.

本研究提出了一种基于 IACycleGAN 模型的人脸照片 - 草图合成算法，采用感知损失监督图像生成网络以改进 CycleGAN，在关键的面部区域（如眼睛和鼻子）的合成过程中更加注重识别，同时还使用合成样本和真实样本的三元组损失增强了识别模型，在 CUFS 和 CUFSF 数据库上的实验结果表明，所提出的方法在合成图像质量和草图 - 照片识别精度方面均优于多个现有的先进方法。