Recent advances in 3D avatar generation have gained significant attentions.
These breakthroughs aim to produce more realistic animatable avatars, narrowing
the gap between virtual and real-world experiences. Most of existing works
employ Score Distillation Sampling (SDS) loss, combined with a differentiable
renderer and text condition, to guide a diffusion model in generating 3D
avatars. However, SDS often generates oversmoothed results with few facial
details, thereby lacking the diversity compared with ancestral sampling. On the
other hand, other works generate 3D avatar from a single image, where the
challenges of unwanted lighting effects, perspective views, and inferior image
quality make them difficult to reliably reconstruct the 3D face meshes with the
aligned complete textures. In this paper, we propose a novel 3D avatar
generation approach termed UltrAvatar with enhanced fidelity of geometry, and
superior quality of physically based rendering (PBR) textures without unwanted
lighting. To this end, the proposed approach presents a diffuse color
extraction model and an authenticity guided texture diffusion model. The former
removes the unwanted lighting effects to reveal true diffuse colors so that the
generated avatars can be rendered under various lighting conditions. The latter
follows two gradient-based guidances for generating PBR textures to render
diverse face-identity features and details better aligning with 3D mesh
geometry. We demonstrate the effectiveness and robustness of the proposed
method, outperforming the state-of-the-art methods by a large margin in the
experiments.

提出了一种新颖的 3D 人物生成方法 UltrAvatar，通过增强几何保真度和质感材质的质量来生成更真实且具有物理基础渲染（PBR）效果的 3D 人物模型，克服了光照影响和获得更多面部细节的挑战，并在实验中展示了该方法的有效性和鲁棒性。

UltrAvatar: 逼真可动的三维头像扩散模型与指导纹理

UltrAvatar: A Realistic Animatable 3D Avatar Diffusion Model with  Authenticity Guided Textures

Recent advancements in 3D avatar generation excel with multi-view supervision
for photorealistic models. However, monocular counterparts lag in quality
despite broader applicability. We propose ReCaLab to close this gap. ReCaLab is
a fully-differentiable pipeline that learns high-fidelity 3D human avatars from
just a single RGB video. A pose-conditioned deformable NeRF is optimized to
volumetrically represent a human subject in canonical T-pose. The canonical
representation is then leveraged to efficiently associate viewpoint-agnostic
textures using 2D-3D correspondences. This enables to separately generate
albedo and shading which jointly compose an RGB prediction. The design allows
to control intermediate results for human pose, body shape, texture, and
lighting with text prompts. An image-conditioned diffusion model thereby helps
to animate appearance and pose of the 3D avatar to create video sequences with
previously unseen human motion. Extensive experiments show that ReCaLab
outperforms previous monocular approaches in terms of image quality for image
synthesis tasks. ReCaLab even outperforms multi-view methods that leverage up
to 19x more synchronized videos for the task of novel pose rendering. Moreover,
natural language offers an intuitive user interface for creative manipulation
of 3D human avatars.

ReCaLab 是一个可完全区分的流程，通过学习高保真度的三维人体化身，仅从单一的 RGB 视频中生成。

现实的画布，语言的画笔：从单目视频中塑造 3D 头像

Reality's Canvas, Language's Brush: Crafting 3D Avatars from Monocular  Video

3D avatar creation plays a crucial role in the digital age. However, the
whole production process is prohibitively time-consuming and labor-intensive.
To democratize this technology to a larger audience, we propose AvatarCLIP, a
zero-shot text-driven framework for 3D avatar generation and animation. Unlike
professional software that requires expert knowledge, AvatarCLIP empowers
layman users to customize a 3D avatar with the desired shape and texture, and
drive the avatar with the described motions using solely natural languages. Our
key insight is to take advantage of the powerful vision-language model CLIP for
supervising neural human generation, in terms of 3D geometry, texture and
animation. Specifically, driven by natural language descriptions, we initialize
3D human geometry generation with a shape VAE network. Based on the generated
3D human shapes, a volume rendering model is utilized to further facilitate
geometry sculpting and texture generation. Moreover, by leveraging the priors
learned in the motion VAE, a CLIP-guided reference-based motion synthesis
method is proposed for the animation of the generated 3D avatar. Extensive
qualitative and quantitative experiments validate the effectiveness and
generalizability of AvatarCLIP on a wide range of avatars. Remarkably,
AvatarCLIP can generate unseen 3D avatars with novel animations, achieving
superior zero-shot capability.

提出了一种使用自然语言驱动的零样本文本框架 --AvatarCLIP，用于三维头像的生成和动画制作，并利用 CLIP 来监督神经人类的生成，实现了超强的零样本能力。