Novel view synthesis of dynamic scenes is becoming important in various applications, including augmented and virtual reality. We propose a novel 4D Gaussian Splatting (4DGS) algorithm for dynamic scenes from casually recorded monocular videos. To overcome the overfitting problem of existing work for these real-world videos, we introduce an uncertainty-aware regularization that identifies uncertain regions with few observations and selectively imposes additional priors based on diffusion models and depth smoothness on such regions. This approach improves both the performance of novel view synthesis and the quality of training image reconstruction. We also identify the initialization problem of 4DGS in fast-moving dynamic regions, where the Structure from Motion (SfM) algorithm fails to provide reliable 3D landmarks. To initialize Gaussian primitives in such regions, we present a dynamic region densification method using the estimated depth maps and scene flow. Our experiments show that the proposed method improves the performance of 4DGS reconstruction from a video captured by a handheld monocular camera and also exhibits promising results in few-shot static scene reconstruction.

本研究解决了现有动态场景合成算法在处理真实世界录制的单目视频时的过拟合问题。我们提出了一种新的4D高斯溅射算法，并引入不确定性感知正则化，能够在不确定区域施加额外的先验知识，从而提高新视图合成及训练图像重建的性能。实验结果表明，该方法在手持单目摄像机捕获的视频重建中显著提升了性能，并在少量静态场景重建中表现出良好效果。

带有不确定性感知正则化的动态场景4D高斯溅射