Neural Radiance Fields (NeRFs) aim to synthesize novel views of objects and
scenes, given the object-centric camera views with large overlaps. However, we
conjugate that this paradigm does not fit the nature of the street views that
are collected by many self-driving cars from the large-scale unbounded scenes.
Also, the onboard cameras perceive scenes without much overlapping. Thus,
existing NeRFs often produce blurs, 'floaters' and other artifacts on
street-view synthesis. In this paper, we propose a new street-view NeRF
(S-NeRF) that considers novel view synthesis of both the large-scale background
scenes and the foreground moving vehicles jointly. Specifically, we improve the
scene parameterization function and the camera poses for learning better neural
representations from street views. We also use the the noisy and sparse LiDAR
points to boost the training and learn a robust geometry and reprojection based
confidence to address the depth outliers. Moreover, we extend our S-NeRF for
reconstructing moving vehicles that is impracticable for conventional NeRFs.
Thorough experiments on the large-scale driving datasets (e.g., nuScenes and
Waymo) demonstrate that our method beats the state-of-the-art rivals by
reducing 7% to 40% of the mean-squared error in the street-view synthesis and a
45% PSNR gain for the moving vehicles rendering.

本文提出了一种新的街景神经辐射场模型（S-NeRF），它考虑了大规模背景场景和前景移动车辆的新视角合成，通过改进场景参数化函数和学习更好的神经表示来应对现有 NeRF 模型在街景合成中出现的问题，并使用嘈杂和稀疏的 LiDAR 点来提高训练的鲁棒性和减少深度奇异值出现的情况。我们的 S-NeRF 模型在各种大规模驾驶数据集上的实验证明了它优于现有最先进方法在街景合成中减少了 7％至 40％的均方误差，并在移动车辆渲染的 PSNR 方面获得了 45％的增益。