We introduce Gaussian-Flow, a novel point-based approach for fast dynamic
scene reconstruction and real-time rendering from both multi-view and monocular
videos. In contrast to the prevalent NeRF-based approaches hampered by slow
training and rendering speeds, our approach harnesses recent advancements in
point-based 3D Gaussian Splatting (3DGS). Specifically, a novel Dual-Domain
Deformation Model (DDDM) is proposed to explicitly model attribute deformations
of each Gaussian point, where the time-dependent residual of each attribute is
captured by a polynomial fitting in the time domain, and a Fourier series
fitting in the frequency domain. The proposed DDDM is capable of modeling
complex scene deformations across long video footage, eliminating the need for
training separate 3DGS for each frame or introducing an additional implicit
neural field to model 3D dynamics. Moreover, the explicit deformation modeling
for discretized Gaussian points ensures ultra-fast training and rendering of a
4D scene, which is comparable to the original 3DGS designed for static 3D
reconstruction. Our proposed approach showcases a substantial efficiency
improvement, achieving a $5\times$ faster training speed compared to the
per-frame 3DGS modeling. In addition, quantitative results demonstrate that the
proposed Gaussian-Flow significantly outperforms previous leading methods in
novel view rendering quality. Project page:
this https URL

我们引入了高斯流（Gaussian-Flow），一种用于从多视图和单目视频中快速动态场景重建和实时渲染的新型基于点的方法。与以往基于 NeRF 的方法不同，这种方法利用了最新的基于点的 3D 高斯喷洒（3DGS）技术，通过引入新颖的双域变形模型（DDDM）来显式建模每个高斯点的属性变形，从而消除了为每个帧训练单独的 3DGS 或引入额外的隐式神经场来建模 3D 动态的需求。此外，对离散化高斯点进行显式变形建模确保了对 4D 场景的超快速训练和渲染，与用于静态 3D 重建的原始 3DGS 相当。我们的方法在效率上取得了显著的改进，训练速度比每帧 3DGS 建模快 5 倍，并且定量结果表明，我们提出的高斯流在新视图渲染质量上明显优于以往的主流方法。