Face sketch synthesis has made great progress in the past few years. Recent
methods based on deep neural networks are able to generate high quality
sketches from face photos. However, due to the lack of training data
(photo-sketch pairs), none of such deep learning based methods can be applied
successfully to face photos in the wild. In this paper, we propose a
semi-supervised deep learning architecture which extends face sketch synthesis
to handle face photos in the wild by exploiting additional face photos in
training. Instead of supervising the network with ground truth sketches, we
first perform patch matching in feature space between the input photo and
photos in a small reference set of photo-sketch pairs. We then compose a pseudo
sketch feature representation using the corresponding sketch feature patches to
supervise our network. With the proposed approach, we can train our networks
using a small reference set of photo-sketch pairs together with a large face
photo dataset without ground truth sketches. Experiments show that our method
achieve state-of-the-art performance both on public benchmarks and face photos
in the wild. Codes are available at
this https URL

本文提出了一种半监督深度学习架构，扩展了面部素描合成以处理现实中的面部照片，该方法使用小的参考照片集中的照片与输入照片进行补丁匹配，并使用相应的素描特征补丁组成伪素描特征表示来监督网络。通过该方法，我们可以使用参考照片集和大型面部照片数据集对网络进行训练，并取得公共基准测试和野外面部照片方面最先进的表现。