We present a first step towards 4D (3D and time) human video stylization,
which addresses style transfer, novel view synthesis and human animation within
a unified framework. While numerous video stylization methods have been
developed, they are often restricted to rendering images in specific viewpoints
of the input video, lacking the capability to generalize to novel views and
novel poses in dynamic scenes. To overcome these limitations, we leverage
Neural Radiance Fields (NeRFs) to represent videos, conducting stylization in
the rendered feature space. Our innovative approach involves the simultaneous
representation of both the human subject and the surrounding scene using two
NeRFs. This dual representation facilitates the animation of human subjects
across various poses and novel viewpoints. Specifically, we introduce a novel
geometry-guided tri-plane representation, significantly enhancing feature
representation robustness compared to direct tri-plane optimization. Following
the video reconstruction, stylization is performed within the NeRFs' rendered
feature space. Extensive experiments demonstrate that the proposed method
strikes a superior balance between stylized textures and temporal coherence,
surpassing existing approaches. Furthermore, our framework uniquely extends its
capabilities to accommodate novel poses and viewpoints, making it a versatile
tool for creative human video stylization.

本文提出了一种 4D（3D 和时间）人体视频风格化的方法，该方法在同一个框架内解决了风格转移、新视角合成和人体动画的问题。通过利用神经辐射场（NeRFs）来表示视频，我们可以在渲染特征空间中进行风格化处理，从而实现对于新视角和新姿势的泛化。实验证明，该方法在风格纹理和时间连贯性方面取得了出色的平衡，优于现有方法，并可以适应新的姿势和视角，成为创作人体视频风格的多功能工具。