Video generative models are receiving particular attention given their
ability to generate realistic and imaginative frames. Besides, these models are
also observed to exhibit strong 3D consistency, significantly enhancing their
potential to act as world simulators. In this work, we present Vidu4D, a novel
reconstruction model that excels in accurately reconstructing 4D (i.e.,
sequential 3D) representations from single generated videos, addressing
challenges associated with non-rigidity and frame distortion. This capability
is pivotal for creating high-fidelity virtual contents that maintain both
spatial and temporal coherence. At the core of Vidu4D is our proposed Dynamic
Gaussian Surfels (DGS) technique. DGS optimizes time-varying warping functions
to transform Gaussian surfels (surface elements) from a static state to a
dynamically warped state. This transformation enables a precise depiction of
motion and deformation over time. To preserve the structural integrity of
surface-aligned Gaussian surfels, we design the warped-state geometric
regularization based on continuous warping fields for estimating normals.
Additionally, we learn refinements on rotation and scaling parameters of
Gaussian surfels, which greatly alleviates texture flickering during the
warping process and enhances the capture of fine-grained appearance details.
Vidu4D also contains a novel initialization state that provides a proper start
for the warping fields in DGS. Equipping Vidu4D with an existing video
generative model, the overall framework demonstrates high-fidelity text-to-4D
generation in both appearance and geometry.

本论文介绍了一种名为 Vidu4D 的重建模型，通过使用动态高斯 Surfels 技术实现从单个生成的视频中准确重建 4D 表示，解决了非刚性运动和帧畸变等挑战，从而创建具有空间和时间一致性的高保真虚拟内容，并在外观和几何上展示了高保真的文本至 4D 生成。

Vidu4D：使用动态高斯黎曼表面对单一生成视频进行高保真度 4D 重建

Vidu4D: Single Generated Video to High-Fidelity 4D Reconstruction with  Dynamic Gaussian Surfels

Road surface reconstruction plays a crucial role in autonomous driving, which
can be used for road lane perception and autolabeling tasks. Recently,
mesh-based road surface reconstruction algorithms show promising reconstruction
results. However, these mesh-based methods suffer from slow speed and poor
rendering quality. In contrast, the 3D Gaussian Splatting (3DGS) shows superior
rendering speed and quality. Although 3DGS employs explicit Gaussian spheres to
represent the scene, it lacks the ability to directly represent the geometric
information of the scene. To address this limitation, we propose a novel
large-scale road surface reconstruction approach based on 2D Gaussian Splatting
(2DGS), named RoGS. The geometric shape of the road is explicitly represented
using 2D Gaussian surfels, where each surfel stores color, semantics, and
geometric information. Compared to Gaussian spheres, the Gaussian surfels
aligns more closely with the physical reality of the road. Distinct from
previous initialization methods that rely on point clouds for Gaussian spheres,
we introduce a trajectory-based initialization for Gaussian surfels. Thanks to
the explicit representation of the Gaussian surfels and a good initialization,
our method achieves a significant acceleration while improving reconstruction
quality. We achieve excellent results in reconstruction of roads surfaces in a
variety of challenging real-world scenes.

基于 2D 高斯点云散离（2DGS）的 RoGS 方法通过显式地表示道路的几何形状，以及使用基于轨迹的初始化方法，实现了在提升重建质量的同时大幅加速道路表面重建的目标。

RoGS：基于 2D 高斯散布的大规模路面重建

RoGS: Large Scale Road Surface Reconstruction based on 2D Gaussian  Splatting

We propose a novel point-based representation, Gaussian surfels, to combine
the advantages of the flexible optimization procedure in 3D Gaussian points and
the surface alignment property of surfels. This is achieved by directly setting
the z-scale of 3D Gaussian points to 0, effectively flattening the original 3D
ellipsoid into a 2D ellipse. Such a design provides clear guidance to the
optimizer. By treating the local z-axis as the normal direction, it greatly
improves optimization stability and surface alignment. While the derivatives to
the local z-axis computed from the covariance matrix are zero in this setting,
we design a self-supervised normal-depth consistency loss to remedy this issue.
Monocular normal priors and foreground masks are incorporated to enhance the
quality of the reconstruction, mitigating issues related to highlights and
background. We propose a volumetric cutting method to aggregate the information
of Gaussian surfels so as to remove erroneous points in depth maps generated by
alpha blending. Finally, we apply screened Poisson reconstruction method to the
fused depth maps to extract the surface mesh. Experimental results show that
our method demonstrates superior performance in surface reconstruction compared
to state-of-the-art neural volume rendering and point-based rendering methods.

我们提出了一种新颖的基于点的表示方法，高斯 surfels，通过直接将 3D 高斯点的 z 缩放设置为 0，将原始的 3D 椭球形状变为 2D 椭圆，进而提供了对优化过程的明确指导，从而在优化稳定性和表面对齐方面取得了显著的改进。我们还设计了一个自监督的法线深度一致性损失，以解决由于该设置使得从协方差矩阵计算得到的法线方向导数为零的问题。通过引入单目法线和前景蒙版，我们提高了重建质量，并缓解了与高光和背景相关的问题。此外，我们提出了一种用于聚合高斯 surfels 信息以去除通过 alpha 混合生成的深度图中的错误点的容积切割方法。最后，我们应用屏蔽泊松重建方法提取了表面网格。实验证明，与最先进的神经体积渲染和基于点的渲染方法相比，我们的方法在表面重建方面表现出卓越的性能。