While recent works have achieved great success on one-shot 3D common object
generation, high quality and fidelity 3D head generation from a single image
remains a great challenge. Previous text-based methods for generating 3D heads
were limited by text descriptions and image-based methods struggled to produce
high-quality head geometry. To handle this challenging problem, we propose a
novel framework, Portrait3D, to generate high-quality 3D heads while preserving
their identities. Our work incorporates the identity information of the
portrait image into three parts: 1) geometry initialization, 2) geometry
sculpting, and 3) texture generation stages. Given a reference portrait image,
we first align the identity features with text features to realize ID-aware
guidance enhancement, which contains the control signals representing the face
information. We then use the canny map, ID features of the portrait image, and
a pre-trained text-to-normal/depth diffusion model to generate ID-aware
geometry supervision, and 3D-GAN inversion is employed to generate ID-aware
geometry initialization. Furthermore, with the ability to inject identity
information into 3D head generation, we use ID-aware guidance to calculate
ID-aware Score Distillation (ISD) for geometry sculpting. For texture
generation, we adopt the ID Consistent Texture Inpainting and Refinement which
progressively expands the view for texture inpainting to obtain an
initialization UV texture map. We then use the id-aware guidance to provide
image-level supervision for noisy multi-view images to obtain a refined texture
map. Extensive experiments demonstrate that we can generate high-quality 3D
heads with accurate geometry and texture from single in-the-wild portrait
images. The project page is at this https URL

提出了 Portrait3D 框架，通过将肖像图像的身份信息整合到几何初始化、几何塑造和纹理生成阶段中，实现了从单幅肖像图像生成高质量 3D 头像的方法。