In this paper, we propose a 3D geometry-aware deformable Gaussian Splatting
method for dynamic view synthesis. Existing neural radiance fields (NeRF) based
solutions learn the deformation in an implicit manner, which cannot incorporate
3D scene geometry. Therefore, the learned deformation is not necessarily
geometrically coherent, which results in unsatisfactory dynamic view synthesis
and 3D dynamic reconstruction. Recently, 3D Gaussian Splatting provides a new
representation of the 3D scene, building upon which the 3D geometry could be
exploited in learning the complex 3D deformation. Specifically, the scenes are
represented as a collection of 3D Gaussian, where each 3D Gaussian is optimized
to move and rotate over time to model the deformation. To enforce the 3D scene
geometry constraint during deformation, we explicitly extract 3D geometry
features and integrate them in learning the 3D deformation. In this way, our
solution achieves 3D geometry-aware deformation modeling, which enables
improved dynamic view synthesis and 3D dynamic reconstruction. Extensive
experimental results on both synthetic and real datasets prove the superiority
of our solution, which achieves new state-of-the-art performance.
The project is available at this https URL

本文提出了一种 3D 几何感知可变形高斯喷洒方法，用于动态视图合成。我们的解决方案通过显式提取和融合 3D 几何特征来实现 3D 几何感知的变形建模，从而实现改进的动态视图合成和 3D 动态重建。