We present FaceVerse, a fine-grained 3D Neural Face Model, which is built
from hybrid East Asian face datasets containing 60K fused RGB-D images and 2K
high-fidelity 3D head scan models. A novel coarse-to-fine structure is proposed
to take better advantage of our hybrid dataset. In the coarse module, we
generate a base parametric model from large-scale RGB-D images, which is able
to predict accurate rough 3D face models in different genders, ages, etc. Then
in the fine module, a conditional StyleGAN architecture trained with
high-fidelity scan models is introduced to enrich elaborate facial geometric
and texture details. Note that different from previous methods, our base and
detailed modules are both changeable, which enables an innovative application
of adjusting both the basic attributes and the facial details of 3D face
models. Furthermore, we propose a single-image fitting framework based on
differentiable rendering. Rich experiments show that our method outperforms the
state-of-the-art methods.

提出了 FaceVerse，这是一个精细的 3D 神经人脸模型，利用混合的东亚人脸数据集建立，通过基于大规模 RGB-D 图像生成基本参数模型，再利用高保真的 3D 头部扫描模型选用 StyleGAN 结构进行精细调整，同时结合基本属性和面部细节进行创新性的调节，实现了单张图像拟合，相比其他最先进的方法表现更好。