Recent text-to-3D methods employing diffusion models have made significant
advancements in 3D human generation. However, these approaches face challenges
due to the limitations of the text-to-image diffusion model, which lacks an
understanding of 3D structures. Consequently, these methods struggle to achieve
high-quality human generation, resulting in smooth geometry and cartoon-like
appearances. In this paper, we observed that fine-tuning text-to-image
diffusion models with normal maps enables their adaptation into text-to-normal
diffusion models, which enhances the 2D perception of 3D geometry while
preserving the priors learned from large-scale datasets. Therefore, we propose
HumanNorm, a novel approach for high-quality and realistic 3D human generation
by learning the normal diffusion model including a normal-adapted diffusion
model and a normal-aligned diffusion model. The normal-adapted diffusion model
can generate high-fidelity normal maps corresponding to prompts with
view-dependent text. The normal-aligned diffusion model learns to generate
color images aligned with the normal maps, thereby transforming physical
geometry details into realistic appearance. Leveraging the proposed normal
diffusion model, we devise a progressive geometry generation strategy and
coarse-to-fine texture generation strategy to enhance the efficiency and
robustness of 3D human generation. Comprehensive experiments substantiate our
method's ability to generate 3D humans with intricate geometry and realistic
appearances, significantly outperforming existing text-to-3D methods in both
geometry and texture quality. The project page of HumanNorm is
this https URL

通过使用法线贴图对文本到图像传播模型进行微调，使其能够适应文本到法线传播模型，从而提高对三维几何体的二维感知，同时保留从大规模数据集中学习到的先验知识，从而实现高质量和逼真的三维人体生成。