In this study, we introduce a methodology for human image animation by
leveraging a 3D human parametric model within a latent diffusion framework to
enhance shape alignment and motion guidance in curernt human generative
techniques. The methodology utilizes the SMPL(Skinned Multi-Person Linear)
model as the 3D human parametric model to establish a unified representation of
body shape and pose. This facilitates the accurate capture of intricate human
geometry and motion characteristics from source videos. Specifically, we
incorporate rendered depth images, normal maps, and semantic maps obtained from
SMPL sequences, alongside skeleton-based motion guidance, to enrich the
conditions to the latent diffusion model with comprehensive 3D shape and
detailed pose attributes. A multi-layer motion fusion module, integrating
self-attention mechanisms, is employed to fuse the shape and motion latent
representations in the spatial domain. By representing the 3D human parametric
model as the motion guidance, we can perform parametric shape alignment of the
human body between the reference image and the source video motion.
Experimental evaluations conducted on benchmark datasets demonstrate the
methodology's superior ability to generate high-quality human animations that
accurately capture both pose and shape variations. Furthermore, our approach
also exhibits superior generalization capabilities on the proposed wild
dataset. Project page: this https URL

通过在潜在扩散框架中利用 3D 人体参数模型，介绍了一种人像动画的方法，以增强当前人体生成技术中的形状对齐和运动引导，通过捕捉源视频中的复杂人体几何和运动特征，利用深度图像、法线图和语义图来丰富潜在扩散模型的综合三维形状和详细姿势属性的条件，通过多层运动融合模块在空间域中融合形状和运动潜在表示，并通过以 3D 人体参数模型作为运动引导来执行参变形对齐，实验证明该方法能够生成准确捕捉姿势和形状变化的高质量人体动画，并且在所提出的数据集上具有优越的泛化能力。