In high-end visual effects pipelines, a customized (and expensive) light
stage system is (typically) used to scan an actor in order to acquire both
geometry and texture for various expressions. Aiming towards democratization,
we propose a novel pipeline for obtaining geometry and texture as well as
enough expression information to build a customized person-specific animation
rig without using a light stage or any other high-end hardware (or manual
cleanup). A key novel idea consists of warping real-world images to align with
the geometry of a template avatar and subsequently projecting the warped image
into the template avatar's texture; importantly, this allows us to leverage
baked-in real-world lighting/texture information in order to create surrogate
facial features (and bridge the domain gap) for the sake of geometry
reconstruction. Not only can our method be used to obtain a neutral expression
geometry and de-lit texture, but it can also be used to improve avatars after
they have been imported into an animation system (noting that such imports tend
to be lossy, while also hallucinating various features). Since a default
animation rig will contain template expressions that do not correctly
correspond to those of a particular individual, we use a Simon Says approach to
capture various expressions and build a person-specific animation rig (that
moves like they do). Our aforementioned warping/projection method has high
enough efficacy to reconstruct geometry corresponding to each expressions.

该研究提出了一种新的方法，使用现实世界的图像信息来重建几何形状，进而创建个性化的动画系统，从而大幅降低成本和硬件要求。

面部可动态建模的民主化

Democratizing the Creation of Animatable Facial Avatars

The ability to generate diverse 3D articulated head avatars is vital to a
plethora of applications, including augmented reality, cinematography, and
education. Recent work on text-guided 3D object generation has shown great
promise in addressing these needs. These methods directly leverage pre-trained
2D text-to-image diffusion models to generate 3D-multi-view-consistent radiance
fields of generic objects. However, due to the lack of geometry and texture
priors, these methods have limited control over the generated 3D objects,
making it difficult to operate inside a specific domain, e.g., human heads. In
this work, we develop a new approach to text-guided 3D head avatar generation
to address this limitation. Our framework directly operates on the geometry and
texture of an articulable 3D morphable model (3DMM) of a head, and introduces
novel optimization procedures to update the geometry and texture while keeping
the 2D and 3D facial features aligned. The result is a 3D head avatar that is
consistent with the text description and can be readily articulated using the
deformation model of the 3DMM. We show that our diffusion-based articulated
head avatars outperform state-of-the-art approaches for this task. The latter
are typically based on CLIP, which is known to provide limited diversity of
generation and accuracy for 3D object generation.

本文提出了一种新的基于 3D 可塑模型（3DMM）的文本引导头部生成方法，利用扩散模型生成多视图一致性辐射场的同时更新几何和纹理以生成和文本描述一致的 3D 头部化身，以此在一定程度上提高 3D 头部生成的精度和多样性。