We introduce 4D Motion Scaffolds (MoSca), a neural information processing
system designed to reconstruct and synthesize novel views of dynamic scenes
from monocular videos captured casually in the wild. To address such a
challenging and ill-posed inverse problem, we leverage prior knowledge from
foundational vision models, lift the video data to a novel Motion Scaffold
(MoSca) representation, which compactly and smoothly encodes the underlying
motions / deformations. The scene geometry and appearance are then disentangled
from the deformation field, and are encoded by globally fusing the Gaussians
anchored onto the MoSca and optimized via Gaussian Splatting. Additionally,
camera poses can be seamlessly initialized and refined during the dynamic
rendering process, without the need for other pose estimation tools.
Experiments demonstrate state-of-the-art performance on dynamic rendering
benchmarks.

我们介绍了 4D 动作支架（MoSca），这是一个神经信息处理系统，旨在从在野外随意捕获的单目视频重建和合成动态场景的新视图。