Photorealistic 3D reconstruction of street scenes is a critical technique for
developing real-world simulators for autonomous driving. Despite the efficacy
of Neural Radiance Fields (NeRF) for driving scenes, 3D Gaussian Splatting
(3DGS) emerges as a promising direction due to its faster speed and more
explicit representation. However, most existing street 3DGS methods require
tracked 3D vehicle bounding boxes to decompose the static and dynamic elements
for effective reconstruction, limiting their applications for in-the-wild
scenarios. To facilitate efficient 3D scene reconstruction without costly
annotations, we propose a self-supervised street Gaussian
($\textit{S}^3$Gaussian) method to decompose dynamic and static elements from
4D consistency. We represent each scene with 3D Gaussians to preserve the
explicitness and further accompany them with a spatial-temporal field network
to compactly model the 4D dynamics. We conduct extensive experiments on the
challenging Waymo-Open dataset to evaluate the effectiveness of our method. Our
$\textit{S}^3$Gaussian demonstrates the ability to decompose static and dynamic
scenes and achieves the best performance without using 3D annotations. Code is
available at: this https URL

通过自我监督学习的方法，在不使用 3D 注释的情况下，我们提出了一种自适应街景高斯方法来分解静态和动态元素并取得最佳性能。

$	extit {S}^3$Gaussian：自主驾驶的自我监督街道高斯分布

$\textit{S}^3$Gaussian: Self-Supervised Street Gaussians for Autonomous  Driving

Photorealistic simulation plays a crucial role in applications such as
autonomous driving, where advances in neural radiance fields (NeRFs) may allow
better scalability through the automatic creation of digital 3D assets.
However, reconstruction quality suffers on street scenes due to largely
collinear camera motions and sparser samplings at higher speeds. On the other
hand, the application often demands rendering from camera views that deviate
from the inputs to accurately simulate behaviors like lane changes. In this
paper, we propose several insights that allow a better utilization of Lidar
data to improve NeRF quality on street scenes. First, our framework learns a
geometric scene representation from Lidar, which is fused with the implicit
grid-based representation for radiance decoding, thereby supplying stronger
geometric information offered by explicit point cloud. Second, we put forth a
robust occlusion-aware depth supervision scheme, which allows utilizing
densified Lidar points by accumulation. Third, we generate augmented training
views from Lidar points for further improvement. Our insights translate to
largely improved novel view synthesis under real driving scenes.

利用 Lidar 数据改善街景下基于神经辐射场的光线追踪和视图合成技术。

DiL-NeRF：透视街景中的激光雷达神经辐射场探究

DiL-NeRF: Delving into Lidar for Neural Radiance Field on Street Scenes

Novel View Synthesis (NVS) for street scenes play a critical role in the
autonomous driving simulation. The current mainstream technique to achieve it
is neural rendering, such as Neural Radiance Fields (NeRF) and 3D Gaussian
Splatting (3DGS). Although thrilling progress has been made, when handling
street scenes, current methods struggle to maintain rendering quality at the
viewpoint that deviates significantly from the training viewpoints. This issue
stems from the sparse training views captured by a fixed camera on a moving
vehicle. To tackle this problem, we propose a novel approach that enhances the
capacity of 3DGS by leveraging prior from a Diffusion Model along with
complementary multi-modal data. Specifically, we first fine-tune a Diffusion
Model by adding images from adjacent frames as condition, meanwhile exploiting
depth data from LiDAR point clouds to supply additional spatial information.
Then we apply the Diffusion Model to regularize the 3DGS at unseen views during
training. Experimental results validate the effectiveness of our method
compared with current state-of-the-art models, and demonstrate its advance in
rendering images from broader views.

用新颖的方法结合扩散模型和多模态数据以提高三维高斯飞溅对未见视角的渲染效果，有效增强了自动驾驶模拟中街景的新视角合成技术。