We present an integral framework for training sketch simplification networks
that convert challenging rough sketches into clean line drawings. Our approach
augments a simplification network with a discriminator network, training both
networks jointly so that the discriminator network discerns whether a line
drawing is a real training data or the output of the simplification network,
which in turn tries to fool it. This approach has two major advantages. First,
because the discriminator network learns the structure in line drawings, it
encourages the output sketches of the simplification network to be more similar
in appearance to the training sketches. Second, we can also train the
simplification network with additional unsupervised data, using the
discriminator network as a substitute teacher. Thus, by adding only rough
sketches without simplified line drawings, or only line drawings without the
original rough sketches, we can improve the quality of the sketch
simplification. We show how our framework can be used to train models that
significantly outperform the state of the art in the sketch simplification
task, despite using the same architecture for inference. We additionally
present an approach to optimize for a single image, which improves accuracy at
the cost of additional computation time. Finally, we show that, using the same
framework, it is possible to train the network to perform the inverse problem,
i.e., convert simple line sketches into pencil drawings, which is not possible
using the standard mean squared error loss. We validate our framework with two
user tests, where our approach is preferred to the state of the art in sketch
simplification 92.3% of the time and obtains 1.2 more points on a scale of 1 to
5.

本文提出了一个基于神经网络和非监督学习的草图简化模型，结合判别器网络和辅助未标注数据，使得输出草图更接近训练数据，并可以用于优化单张图像和实现草图转铅笔画的逆问题，并在两个用户测试中取得了优秀的效果。