We introduce a highly robust GAN-based framework for digitizing a normalized
3D avatar of a person from a single unconstrained photo. While the input image
can be of a smiling person or taken in extreme lighting conditions, our method
can reliably produce a high-quality textured model of a person's face in
neutral expression and skin textures under diffuse lighting condition.
Cutting-edge 3D face reconstruction methods use non-linear morphable face
models combined with GAN-based decoders to capture the likeness and details of
a person but fail to produce neutral head models with unshaded albedo textures
which is critical for creating relightable and animation-friendly avatars for
integration in virtual environments. The key challenges for existing methods to
work is the lack of training and ground truth data containing normalized 3D
faces. We propose a two-stage approach to address this problem. First, we adopt
a highly robust normalized 3D face generator by embedding a non-linear
morphable face model into a StyleGAN2 network. This allows us to generate
detailed but normalized facial assets. This inference is then followed by a
perceptual refinement step that uses the generated assets as regularization to
cope with the limited available training samples of normalized faces. We
further introduce a Normalized Face Dataset, which consists of a combination
photogrammetry scans, carefully selected photographs, and generated fake people
with neutral expressions in diffuse lighting conditions. While our prepared
dataset contains two orders of magnitude less subjects than cutting edge
GAN-based 3D facial reconstruction methods, we show that it is possible to
produce high-quality normalized face models for very challenging unconstrained
input images, and demonstrate superior performance to the current
state-of-the-art.

提出了一种高度稳健的基于 GAN 的框架，可从单个非约束性照片中数字化人物的标准化 3D 头像，并通过采用高度稳健的标准化 3D 面部生成器以及感知细化步骤来生成高质量的标准化面部模型。