This paper introduces a novel representation of volumetric videos for
real-time view synthesis of dynamic scenes. Recent advances in neural scene
representations demonstrate their remarkable capability to model and render
complex static scenes, but extending them to represent dynamic scenes is not
straightforward due to their slow rendering speed or high storage cost. To
solve this problem, our key idea is to represent the radiance field of each
frame as a set of shallow MLP networks whose parameters are stored in 2D grids,
called MLP maps, and dynamically predicted by a 2D CNN decoder shared by all
frames. Representing 3D scenes with shallow MLPs significantly improves the
rendering speed, while dynamically predicting MLP parameters with a shared 2D
CNN instead of explicitly storing them leads to low storage cost. Experiments
show that the proposed approach achieves state-of-the-art rendering quality on
the NHR and ZJU-MoCap datasets, while being efficient for real-time rendering
with a speed of 41.7 fps for $512 \times 512$ images on an RTX 3090 GPU. The
code is available at this https URL

本文提出了一种用于动态场景实时视角合成的体积视频新表示法，通过使用浅层 MLP 网格和共享的 2D CNN 解码器动态预测 MLP 参数，显著提高了渲染速度并降低了存储成本，实验表明，该方法在渲染质量上实现了最先进的水平，同时能够通过 RTX 3090 GPU 以 41.7fps 的速度对 $512 	imes 512$ 的图像实时渲染。