Synthesizing realistic videos of humans using neural networks has been a popular alternative to the conventional graphics-based rendering pipeline due to its high efficiency. Existing works typically formulate this as an image-to-image translation problem in 2D screen space, which leads to artifacts such as over-smoothing, missing body parts, and temporal instability of fine-scale detail, such as pose-dependent wrinkles in the clothing. In this paper, we propose a novel human video synthesis method that approaches these limiting factors by explicitly disentangling the learning of time-coherent fine-scale details from the embedding of the human in 2D screen space. More specifically, our method relies on the combination of two convolutional neural networks (CNNs). Given the pose information, the first CNN predicts a dynamic texture map that contains time-coherent high-frequency details, and the second CNN conditions the generation of the final video on the temporally coherent output of the first CNN. We demonstrate several applications of our approach, such as human reenactment and novel view synthesis from monocular video, where we show significant improvement over the state of the art both qualitatively and quantitatively.

本文介绍了一种新的人类视频合成方法，通过两个卷积神经网络（CNNs）明确分离2D屏幕空间中的人类嵌入和时间相干的细节，其中第一个CNN预测一个动态纹理映射，第二个CNN条件下第一个CNN的输出生成最终的视频。 该方法被应用于人类重演和单目视频的新视图合成等多个领域，并且在质量和数量方面都有显着的提高。

通过学习动态纹理和视频渲染转换实现神经人类视频渲染