Unsupervised generation of 3D-aware clothed humans with various appearances
and controllable geometries is important for creating virtual human avatars and
other AR/VR applications. Existing methods are either limited to rigid object
modeling, or not generative and thus unable to generate high-quality virtual
humans and animate them. In this work, we propose AvatarGen, the first method
that enables not only geometry-aware clothed human synthesis with high-fidelity
appearances but also disentangled human animation controllability, while only
requiring 2D images for training. Specifically, we decompose the generative 3D
human synthesis into pose-guided mapping and canonical representation with
predefined human pose and shape, such that the canonical representation can be
explicitly driven to different poses and shapes with the guidance of a 3D
parametric human model SMPL. AvatarGen further introduces a deformation network
to learn non-rigid deformations for modeling fine-grained geometric details and
pose-dependent dynamics. To improve the geometry quality of the generated human
avatars, it leverages the signed distance field as geometric proxy, which
allows more direct regularization from the 3D geometric priors of SMPL.
Benefiting from these designs, our method can generate animatable 3D human
avatars with high-quality appearance and geometry modeling, significantly
outperforming previous 3D GANs. Furthermore, it is competent for many
applications, e.g., single-view reconstruction, re-animation, and text-guided
synthesis/editing. Code and pre-trained model will be available at
this http URL

本文提出 AvatarGen 方法，是第一种通过仅使用 2D 图像训练，能够生成高保真度外观和可控几何形状的、解耦式的可控人体动画的方法。