We propose a novel approach for 3D video synthesis that is able to represent
multi-view video recordings of a dynamic real-world scene in a compact, yet
expressive representation that enables high-quality view synthesis and motion
interpolation. Our approach takes the high quality and compactness of static
neural radiance fields in a new direction: to a model-free, dynamic setting. At
the core of our approach is a novel time-conditioned neural radiance field that
represents scene dynamics using a set of compact latent codes. We are able to
significantly boost the training speed and perceptual quality of the generated
imagery by a novel hierarchical training scheme in combination with ray
importance sampling. Our learned representation is highly compact and able to
represent a 10 second 30 FPS multiview video recording by 18 cameras with a
model size of only 28MB. We demonstrate that our method can render
high-fidelity wide-angle novel views at over 1K resolution, even for complex
and dynamic scenes. We perform an extensive qualitative and quantitative
evaluation that shows that our approach outperforms the state of the art.
Project website: this https URL

提出了一种新的 3D 视频合成方法，使用神经辐射场表示动态实景，并使用一种紧凑的表现方式，能够高质量地合成视图和插值运动，其动态场景建模方法在视角合成和动态场景表示方面都表现良好。