We present a novel animatable 3D Gaussian model for rendering high-fidelity
free-view human motions in real time. Compared to existing NeRF-based methods,
the model owns better capability in synthesizing high-frequency details without
the jittering problem across video frames. The core of our model is a novel
augmented 3D Gaussian representation, which attaches each Gaussian with a
learnable code. The learnable code serves as a pose-dependent appearance
embedding for refining the erroneous appearance caused by geometric
transformation of Gaussians, based on which an appearance refinement model is
learned to produce residual Gaussian properties to match the appearance in
target pose. To force the Gaussians to learn the foreground human only without
background interference, we further design a novel alpha loss to explicitly
constrain the Gaussians within the human body. We also propose to jointly
optimize the human joint parameters to improve the appearance accuracy. The
animatable 3D Gaussian model can be learned with shallow MLPs, so new human
motions can be synthesized in real time (66 fps on avarage). Experiments show
that our model has superior performance over NeRF-based methods.

我们提出了一种新颖的可动画化的三维高斯模型，用于实时渲染高保真度的自由视角人体动作。与现有的基于 NeRF 的方法相比，该模型具有更好的能力，在视频帧之间合成高频细节时没有抖动问题。我们的模型的核心是一种新型的增强型三维高斯表示，通过将每个高斯附加一个可学习的代码，该可学习代码作为姿态相关的外观嵌入，用于纠正由于高斯的几何变换而导致的错误外观，在此基础上，学习一种外观细化模型来产生匹配目标姿态的剩余高斯属性。为了使高斯仅学习到人体前景而不受背景干扰，我们进一步设计了一种新型的 Alpha 损失来明确约束高斯在人体内部。我们还提议共同优化人体关节参数以提高外观准确性。此可动画化的三维高斯模型可以通过浅层 MLP 进行学习，因此可以实时合成新的人体动作（平均每秒 66 帧）。实验证明，我们的模型优于基于 NeRF 的方法。