In this paper, we present a method to reconstruct the world and multiple
dynamic humans in 3D from a monocular video input. As a key idea, we represent
both the world and multiple humans via the recently emerging 3D Gaussian
Splatting (3D-GS) representation, enabling to conveniently and efficiently
compose and render them together. In particular, we address the scenarios with
severely limited and sparse observations in 3D human reconstruction, a common
challenge encountered in the real world. To tackle this challenge, we introduce
a novel approach to optimize the 3D-GS representation in a canonical space by
fusing the sparse cues in the common space, where we leverage a pre-trained 2D
diffusion model to synthesize unseen views while keeping the consistency with
the observed 2D appearances. We demonstrate our method can reconstruct
high-quality animatable 3D humans in various challenging examples, in the
presence of occlusion, image crops, few-shot, and extremely sparse
observations. After reconstruction, our method is capable of not only rendering
the scene in any novel views at arbitrary time instances, but also editing the
3D scene by removing individual humans or applying different motions for each
human. Through various experiments, we demonstrate the quality and efficiency
of our methods over alternative existing approaches.

通过单目视频输入的方法，本文提出了一种将世界和多个动态人物在 3D 中进行重构的方法，使用新兴的 3D 高斯涂布（3D-GS）表示法来表达世界和多个人物，解决了 3D 人物重构中常见的观测在 3D 空间中非常有限和稀疏的挑战，并演示了方法在各种具有挑战性的例子中可以重构出高质量的可动画 3D 人物，并且在重构后，该方法可以以任意时间片段渲染场景并编辑 3D 场景。