This paper proposes an image-to-painting translation method that generates
vivid and realistic painting artworks with controllable styles. Different from
previous image-to-image translation methods that formulate the translation as
pixel-wise prediction, we deal with such an artistic creation process in a
vectorized environment and produce a sequence of physically meaningful stroke
parameters that can be further used for rendering. Since a typical vector
render is not differentiable, we design a novel neural renderer which imitates
the behavior of the vector renderer and then frame the stroke prediction as a
parameter searching process that maximizes the similarity between the input and
the rendering output. We explored the zero-gradient problem on parameter
searching and propose to solve this problem from an optimal transportation
perspective. We also show that previous neural renderers have a parameter
coupling problem and we re-design the rendering network with a rasterization
network and a shading network that better handles the disentanglement of shape
and color. Experiments show that the paintings generated by our method have a
high degree of fidelity in both global appearance and local textures. Our
method can be also jointly optimized with neural style transfer that further
transfers visual style from other images. Our code and animated results are
available at https://jiupinjia.github.io/neuralpainter/.

本文提出了一种图像到绘画的翻译方法，采用矢量化环境并产生一系列具有物理意义的绘画参数，然后通过搜索参数最大化输入和渲染输出之间的相似度来框架笔画预测作为参数搜索过程，结果表明生成的画作在全局外观和局部纹理方面具有高度的保真度。