Neural radiance fields have made a remarkable breakthrough in the novel view
synthesis task at the 3D static scene. However, for the 4D circumstance (e.g.,
dynamic scene), the performance of the existing method is still limited by the
capacity of the neural network, typically in a multilayer perceptron network
(MLP). In this paper, we utilize 3D Voxel to model the 4D neural radiance
field, short as V4D, where the 3D voxel has two formats. The first one is to
regularly model the 3D space and then use the sampled local 3D feature with the
time index to model the density field and the texture field by a tiny MLP. The
second one is in look-up tables (LUTs) format that is for the pixel-level
refinement, where the pseudo-surface produced by the volume rendering is
utilized as the guidance information to learn a 2D pixel-level refinement
mapping. The proposed LUTs-based refinement module achieves the performance
gain with little computational cost and could serve as the plug-and-play module
in the novel view synthesis task. Moreover, we propose a more effective
conditional positional encoding toward the 4D data that achieves performance
gain with negligible computational burdens. Extensive experiments demonstrate
that the proposed method achieves state-of-the-art performance at a low
computational cost.

本文提出了一种利用 3D Voxel 建模 4D 神经辐射场的方法，即 V4D, 并使用 LUTs 格式和条件位置编码来改进新视图综合任务的表现，实验表明该方法在低计算成本下取得了最先进的表现。