In high-end visual effects pipelines, a customized (and expensive) light
stage system is (typically) used to scan an actor in order to acquire both
geometry and texture for various expressions. Aiming towards democratization,
we propose a novel pipeline for obtaining geometry and texture as well as
enough expression information to build a customized person-specific animation
rig without using a light stage or any other high-end hardware (or manual
cleanup). A key novel idea consists of warping real-world images to align with
the geometry of a template avatar and subsequently projecting the warped image
into the template avatar's texture; importantly, this allows us to leverage
baked-in real-world lighting/texture information in order to create surrogate
facial features (and bridge the domain gap) for the sake of geometry
reconstruction. Not only can our method be used to obtain a neutral expression
geometry and de-lit texture, but it can also be used to improve avatars after
they have been imported into an animation system (noting that such imports tend
to be lossy, while also hallucinating various features). Since a default
animation rig will contain template expressions that do not correctly
correspond to those of a particular individual, we use a Simon Says approach to
capture various expressions and build a person-specific animation rig (that
moves like they do). Our aforementioned warping/projection method has high
enough efficacy to reconstruct geometry corresponding to each expressions.

该研究提出了一种新的方法，使用现实世界的图像信息来重建几何形状，进而创建个性化的动画系统，从而大幅降低成本和硬件要求。