In this paper, we introduce a novel approach to learn a 3D face model using a
joint-based face rig and a neural skinning network. Thanks to the joint-based
representation, our model enjoys some significant advantages over prior
blendshape-based models. First, it is very compact such that we are orders of
magnitude smaller while still keeping strong modeling capacity. Second, because
each joint has its semantic meaning, interactive facial geometry editing is
made easier and more intuitive. Third, through skinning, our model supports
adding mouth interior and eyes, as well as accessories (hair, eye glasses,
etc.) in a simpler, more accurate and principled way. We argue that because the
human face is highly structured and topologically consistent, it does not need
to be learned entirely from data. Instead we can leverage prior knowledge in
the form of a human-designed 3D face rig to reduce the data dependency, and
learn a compact yet strong face model from only a small dataset (less than one
hundred 3D scans). To further improve the modeling capacity, we train a
skinning weight generator through adversarial learning. Experiments on fitting
high-quality 3D scans (both neutral and expressive), noisy depth images, and
RGB images demonstrate that its modeling capacity is on-par with
state-of-the-art face models, such as FLAME and Facewarehouse, even though the
model is 10 to 20 times smaller. This suggests broad value in both graphics and
vision applications on mobile and edge devices.

本论文提出了一种使用基于关节的面部模型和神经细胞网络来学习三维面部模型的新方法，该模型相比之前的基于混合形态的模型具有更小的体积，更容易进行人脸编辑，更准确地支持添加眼睛，嘴巴和配饰设计，并且仅需一小组数据集，同时支持高质量 3D 扫描和 RGB 图像等广泛适用于图形和视觉应用。