Neural volume rendering enables photo-realistic renderings of a human
performer in free-view, a critical task in immersive VR/AR applications. But
the practice is severely limited by high computational costs in the rendering
process. To solve this problem, we propose the UV Volumes, a new approach that
can render an editable free-view video of a human performer in real-time. It
separates the high-frequency (i.e., non-smooth) human appearance from the 3D
volume, and encodes them into 2D neural texture stacks (NTS). The smooth UV
volumes allow much smaller and shallower neural networks to obtain densities
and texture coordinates in 3D while capturing detailed appearance in 2D NTS.
For editability, the mapping between the parameterized human model and the
smooth texture coordinates allows us a better generalization on novel poses and
shapes. Furthermore, the use of NTS enables interesting applications, e.g.,
retexturing. Extensive experiments on CMU Panoptic, ZJU Mocap, and H36M
datasets show that our model can render 960 x 540 images in 30FPS on average
with comparable photo-realism to state-of-the-art methods. The project and
supplementary materials are available at this https URL

该研究提出了一种名为 UV Volumes 的新方法，利用 2D 的神经纹理堆栈来分离高频（即非平滑）人类外貌和 3D 体积，并用较小和更浅的神经网络在三维中获得密度和纹理坐标，从而实时渲染可编辑的自由视角视频，可以用于 VR / AR 应用，并且具有重新贴图等有趣应用。