Recent advances in 3D avatar generation have gained significant attentions.
These breakthroughs aim to produce more realistic animatable avatars, narrowing
the gap between virtual and real-world experiences. Most of existing works
employ Score Distillation Sampling (SDS) loss, combined with a differentiable
renderer and text condition, to guide a diffusion model in generating 3D
avatars. However, SDS often generates oversmoothed results with few facial
details, thereby lacking the diversity compared with ancestral sampling. On the
other hand, other works generate 3D avatar from a single image, where the
challenges of unwanted lighting effects, perspective views, and inferior image
quality make them difficult to reliably reconstruct the 3D face meshes with the
aligned complete textures. In this paper, we propose a novel 3D avatar
generation approach termed UltrAvatar with enhanced fidelity of geometry, and
superior quality of physically based rendering (PBR) textures without unwanted
lighting. To this end, the proposed approach presents a diffuse color
extraction model and an authenticity guided texture diffusion model. The former
removes the unwanted lighting effects to reveal true diffuse colors so that the
generated avatars can be rendered under various lighting conditions. The latter
follows two gradient-based guidances for generating PBR textures to render
diverse face-identity features and details better aligning with 3D mesh
geometry. We demonstrate the effectiveness and robustness of the proposed
method, outperforming the state-of-the-art methods by a large margin in the
experiments.

提出了一种新颖的 3D 人物生成方法 UltrAvatar，通过增强几何保真度和质感材质的质量来生成更真实且具有物理基础渲染（PBR）效果的 3D 人物模型，克服了光照影响和获得更多面部细节的挑战，并在实验中展示了该方法的有效性和鲁棒性。

UltrAvatar: 逼真可动的三维头像扩散模型与指导纹理

UltrAvatar: A Realistic Animatable 3D Avatar Diffusion Model with  Authenticity Guided Textures

Text-guided 3D face synthesis has achieved remarkable results by leveraging
text-to-image (T2I) diffusion models. However, most existing works focus solely
on the direct generation, ignoring the editing, restricting them from
synthesizing customized 3D faces through iterative adjustments. In this paper,
we propose a unified text-guided framework from face generation to editing. In
the generation stage, we propose a geometry-texture decoupled generation to
mitigate the loss of geometric details caused by coupling. Besides, decoupling
enables us to utilize the generated geometry as a condition for texture
generation, yielding highly geometry-texture aligned results. We further employ
a fine-tuned texture diffusion model to enhance texture quality in both RGB and
YUV space. In the editing stage, we first employ a pre-trained diffusion model
to update facial geometry or texture based on the texts. To enable sequential
editing, we introduce a UV domain consistency preservation regularization,
preventing unintentional changes to irrelevant facial attributes. Besides, we
propose a self-guided consistency weight strategy to improve editing efficacy
while preserving consistency. Through comprehensive experiments, we showcase
our method's superiority in face synthesis. Project page:
this https URL

通过文本引导的三维人脸合成，从生成到编辑的统一框架，通过解耦生成几何和纹理来提高几何细节的生成效果，并利用生成几何作为纹理生成的条件，进一步提升几何与纹理的一致性结果；通过预训练扩散模型来更新面部几何或纹理以实现顺序编辑，并引入 UV 域一致性保持正则化以防止对不相关面部属性的无意更改，并提出自导向一致性权重策略以提高编辑效果与保持一致性；通过全面实验展示了该方法在人脸合成方面的优越性。