We present an efficient neural 3D scene representation for novel-view
synthesis (NVS) in large-scale, dynamic urban areas. Existing works are not
well suited for applications like mixed-reality or closed-loop simulation due
to their limited visual quality and non-interactive rendering speeds. Recently,
rasterization-based approaches have achieved high-quality NVS at impressive
speeds. However, these methods are limited to small-scale, homogeneous data,
i.e. they cannot handle severe appearance and geometry variations due to
weather, season, and lighting and do not scale to larger, dynamic areas with
thousands of images. We propose 4DGF, a neural scene representation that scales
to large-scale dynamic urban areas, handles heterogeneous input data, and
substantially improves rendering speeds. We use 3D Gaussians as an efficient
geometry scaffold while relying on neural fields as a compact and flexible
appearance model. We integrate scene dynamics via a scene graph at global scale
while modeling articulated motions on a local level via deformations. This
decomposed approach enables flexible scene composition suitable for real-world
applications. In experiments, we surpass the state-of-the-art by over 3 dB in
PSNR and more than 200 times in rendering speed.

我们提出一种高效的神经 3D 场景表示方法，用于大规模、动态的城市地区的新视点合成 (NVS)。该方法可以适用于混合现实或闭环仿真等应用，具有较高的可视质量和交互式渲染速度。我们引入 4DGF，一种适用于大规模动态城市地区的神经场景表示方法，它可以处理异构输入数据，并显著提高渲染速度。在实验中，我们的方法的峰值信噪比 (PNSR) 超过最先进方法 3 分贝以上，渲染速度提高了 200 倍以上。