In this paper, we present a practical and robust deep learning solution for the novel view synthesis of complex scenes. In our approach, a continuous scene is represented as a light field, i.e., a set of rays, each of which has a corresponding color. We adopt a 4D parameterization of the light field. We then formulate the light field as a 4D function that maps 4D coordinates to corresponding color values. We train a deep fully connected network to optimize this function. Then, the scene-specific model is used to synthesize novel views. Previous light field approaches usually require dense view sampling to reliably render high-quality novel views. Our method can render novel views by sampling rays and querying the color for each ray from the network directly; thus enabling fast light field rendering with a very sparse set of input images. Our method achieves state-of-the-art novel view synthesis results while maintaining an interactive frame rate.

本文提出了一种深度学习方法，用于复杂场景的新视角合成，通过将3D场景表示为光场，并采用两平面参数化的光场，用4D参数特征化光线，构建4D函数实现映射，最后采用全连接网络优化该函数并合成新视角，同时附加了per-ray depth预测，提高了光场渲染质量，实验表明本方法优于现有技术。

NeuLF: 基于神经网络的 4D 光场高效新视角合成