Reconstructing dynamic 3D scenes from 2D images and generating diverse views
over time is challenging due to scene complexity and temporal dynamics. Despite
advancements in neural implicit models, limitations persist: (i) Inadequate
Scene Structure: Existing methods struggle to reveal the spatial and temporal
structure of dynamic scenes from directly learning the complex 6D plenoptic
function. (ii) Scaling Deformation Modeling: Explicitly modeling scene element
deformation becomes impractical for complex dynamics. To address these issues,
we consider the spacetime as an entirety and propose to approximate the
underlying spatio-temporal 4D volume of a dynamic scene by optimizing a
collection of 4D primitives, with explicit geometry and appearance modeling.
Learning to optimize the 4D primitives enables us to synthesize novel views at
any desired time with our tailored rendering routine. Our model is conceptually
simple, consisting of a 4D Gaussian parameterized by anisotropic ellipses that
can rotate arbitrarily in space and time, as well as view-dependent and
time-evolved appearance represented by the coefficient of 4D spherindrical
harmonics. This approach offers simplicity, flexibility for variable-length
video and end-to-end training, and efficient real-time rendering, making it
suitable for capturing complex dynamic scene motions. Experiments across
various benchmarks, including monocular and multi-view scenarios, demonstrate
our 4DGS model's superior visual quality and efficiency.

从 2D 图像中重建动态 3D 场景并随时间生成多样视角是一项具有挑战性的任务，本研究提出一种基于优化一组 4D 基元的方法，以近似表示动态场景的底层时空 4D 体积，通过特定的渲染程序，制造各种时间的新视角，该方法简单灵活，适用于捕捉复杂动态场景运动，实验结果在多个基准数据集上证明了其优秀的视觉质量和高效性。

实时照片级动态场景表达与渲染：基于四维高斯呈焦

Real-time Photorealistic Dynamic Scene Representation and Rendering with  4D Gaussian Splatting

We propose an efficient approach to exploiting motion information from
consecutive frames of a video sequence to recover the 3D pose of people.
Previous approaches typically compute candidate poses in individual frames and
then link them in a post-processing step to resolve ambiguities. By contrast,
we directly regress from a spatio-temporal volume of bounding boxes to a 3D
pose in the central frame.
We further show that, for this approach to achieve its full potential, it is
essential to compensate for the motion in consecutive frames so that the
subject remains centered. This then allows us to effectively overcome
ambiguities and improve upon the state-of-the-art by a large margin on the
Human3.6m, HumanEva, and KTH Multiview Football 3D human pose estimation
benchmarks.

本文提出了一种高效利用视频序列中连续帧运动信息来恢复人的三维姿态的方法，并通过回归边界框的时空体到中心帧的 3D 姿势来改进现有方法。同时，为了最大限度地发挥这种方法的潜力，本文阐明必须补偿连续帧中的运动，以使被测量者保持中心，从而能够有效地消除歧义并在 Human3.6m、HumanEva 和 KTH Multiview Football 3D 人体姿态估计基准测试中取得显著改进。