In this work, we tackle the task of learning generalizable 3D human Gaussians
from a single image. The main challenge for this task is to recover detailed
geometry and appearance, especially for the unobserved regions. To this end, we
propose single-view generalizable Human Gaussian model (HGM), a
diffusion-guided framework for 3D human modeling from a single image. We design
a diffusion-based coarse-to-fine pipeline, where the diffusion model is adapted
to refine novel-view images rendered from a coarse human Gaussian model. The
refined images are then used together with the input image to learn a refined
human Gaussian model. Although effective in hallucinating the unobserved views,
the approach may generate unrealistic human pose and shapes due to the lack of
supervision. We circumvent this problem by further encoding the geometric
priors from SMPL model. Specifically, we propagate geometric features from SMPL
volume to the predicted Gaussians via sparse convolution and attention
mechanism. We validate our approach on publicly available datasets and
demonstrate that it significantly surpasses state-of-the-art methods in terms
of PSNR and SSIM. Additionally, our method exhibits strong generalization for
in-the-wild images.

从单幅图像中学习可推广的三维人体高斯模型，通过扩散引导的三维人体建模框架，结合几何先验和注意机制，有效地恢复细节几何形状和外观，并具备在自然环境中的强大泛化能力。