Reconstructing dynamic 3D scenes from 2D images and generating diverse views
over time is challenging due to scene complexity and temporal dynamics. Despite
advancements in neural implicit models, limitations persist: (i) Inadequate
Scene Structure: Existing methods struggle to reveal the spatial and temporal
structure of dynamic scenes from directly learning the complex 6D plenoptic
function. (ii) Scaling Deformation Modeling: Explicitly modeling scene element
deformation becomes impractical for complex dynamics. To address these issues,
we consider the spacetime as an entirety and propose to approximate the
underlying spatio-temporal 4D volume of a dynamic scene by optimizing a
collection of 4D primitives, with explicit geometry and appearance modeling.
Learning to optimize the 4D primitives enables us to synthesize novel views at
any desired time with our tailored rendering routine. Our model is conceptually
simple, consisting of a 4D Gaussian parameterized by anisotropic ellipses that
can rotate arbitrarily in space and time, as well as view-dependent and
time-evolved appearance represented by the coefficient of 4D spherindrical
harmonics. This approach offers simplicity, flexibility for variable-length
video and end-to-end training, and efficient real-time rendering, making it
suitable for capturing complex dynamic scene motions. Experiments across
various benchmarks, including monocular and multi-view scenarios, demonstrate
our 4DGS model's superior visual quality and efficiency.

从 2D 图像中重建动态 3D 场景并随时间生成多样视角是一项具有挑战性的任务，本研究提出一种基于优化一组 4D 基元的方法，以近似表示动态场景的底层时空 4D 体积，通过特定的渲染程序，制造各种时间的新视角，该方法简单灵活，适用于捕捉复杂动态场景运动，实验结果在多个基准数据集上证明了其优秀的视觉质量和高效性。