We present an imaging and neural rendering technique that seeks to synthesize
videos of light propagating through a scene from novel, moving camera
viewpoints. Our approach relies on a new ultrafast imaging setup to capture a
first-of-its kind, multi-viewpoint video dataset with picosecond-level temporal
resolution. Combined with this dataset, we introduce an efficient neural volume
rendering framework based on the transient field. This field is defined as a
mapping from a 3D point and 2D direction to a high-dimensional, discrete-time
signal that represents time-varying radiance at ultrafast timescales. Rendering
with transient fields naturally accounts for effects due to the finite speed of
light, including viewpoint-dependent appearance changes caused by light
propagation delays to the camera. We render a range of complex effects,
including scattering, specular reflection, refraction, and diffraction.
Additionally, we demonstrate removing viewpoint-dependent propagation delays
using a time warping procedure, rendering of relativistic effects, and video
synthesis of direct and global components of light transport.

我们提出了一种成像和神经渲染技术，旨在从新颖的移动视角合成通过场景传播的光的视频。我们的方法依赖于一种新的超快成像设备，用皮秒级时间分辨率捕捉首个多视角视频数据集。结合该数据集，我们引入了一种基于瞬态场的高效神经体渲染框架。该场被定义为从三维点和二维方向到表示超快时间尺度下时变辐射度的高维离散时间信号的映射。使用瞬态场进行渲染可以自然地考虑光的有限速度带来的效应，包括由于光传播延迟到相机而引起的视点相关外观变化。我们渲染了一系列复杂效果，包括散射、镜面反射、折射和衍射。此外，我们通过时间扭曲过程演示了去除视点相关传播延迟、相对论效应的渲染以及直接和全局光传输的视频合成。