We present a fully automatic system that can produce high-fidelity,
photo-realistic 3D digital human heads with a consumer RGB-D selfie camera. The
system only needs the user to take a short selfie RGB-D video while rotating
his/her head, and can produce a high quality head reconstruction in less than
30 seconds. Our main contribution is a new facial geometry modeling and
reflectance synthesis procedure that significantly improves the
state-of-the-art. Specifically, given the input video a two-stage frame
selection procedure is first employed to select a few high-quality frames for
reconstruction. Then a differentiable renderer based 3D Morphable Model (3DMM)
fitting algorithm is applied to recover facial geometries from multiview RGB-D
data, which takes advantages of a powerful 3DMM basis constructed with
extensive data generation and perturbation. Our 3DMM has much larger expressive
capacities than conventional 3DMM, allowing us to recover more accurate facial
geometry using merely linear basis. For reflectance synthesis, we present a
hybrid approach that combines parametric fitting and CNNs to synthesize
high-resolution albedo/normal maps with realistic hair/pore/wrinkle details.
Results show that our system can produce faithful 3D digital human faces with
extremely realistic details. The main code and the newly constructed 3DMM basis
is publicly available.

本文介绍一种使用 RGB-D 自拍相机实现高保真、逼真、照片级的 3D 数字人头制作的完全自动化系统。该系统采用不同于传统的面部几何建模和反射合成技术，成功运用了两阶段帧选则过程和可微渲染器的 3DMM 拟合算法来从多视角 RGB-D 数据中恢复面部几何，进而实现人脸表面纹理的抠取与合成，从而制作出了高度真实细节的数字人头。