Recent advances in neural rendering have improved both training and rendering
times by orders of magnitude. While these methods demonstrate state-of-the-art
quality and speed, they are designed for photogrammetry of static scenes and do
not generalize well to freely moving humans in the environment. In this work,
we introduce Human Gaussian Splats (HUGS) that represents an animatable human
together with the scene using 3D Gaussian Splatting (3DGS). Our method takes
only a monocular video with a small number of (50-100) frames, and it
automatically learns to disentangle the static scene and a fully animatable
human avatar within 30 minutes. We utilize the SMPL body model to initialize
the human Gaussians. To capture details that are not modeled by SMPL (e.g.
cloth, hairs), we allow the 3D Gaussians to deviate from the human body model.
Utilizing 3D Gaussians for animated humans brings new challenges, including the
artifacts created when articulating the Gaussians. We propose to jointly
optimize the linear blend skinning weights to coordinate the movements of
individual Gaussians during animation. Our approach enables novel-pose
synthesis of human and novel view synthesis of both the human and the scene. We
achieve state-of-the-art rendering quality with a rendering speed of 60 FPS
while being ~100x faster to train over previous work. Our code will be
announced here: this https URL

通过 3D 高斯模糊表示 animatable 的人物及其所在的场景，以实现对动画人物的新姿势合成和人物和场景的新视图合成，并在提高渲染质量的同时，训练速度更快。