Given a monocular face image as input, 3D face geometry reconstruction aims
to recover a corresponding 3D face mesh. Recently, both optimization-based and
learning-based face reconstruction methods have taken advantage of the emerging
differentiable renderer and shown promising results. However, the
differentiable renderer, mainly based on graphics rules, simplifies the
realistic mechanism of the illumination, reflection, \etc, of the real world,
thus cannot produce realistic images. This brings a lot of domain-shift noise
to the optimization or training process. In this work, we introduce a novel
Generative Adversarial Renderer (GAR) and propose to tailor its inverted
version to the general fitting pipeline, to tackle the above problem.
Specifically, the carefully designed neural renderer takes a face normal map
and a latent code representing other factors as inputs and renders a realistic
face image. Since the GAR learns to model the complicated real-world image,
instead of relying on the simplified graphics rules, it is capable of producing
realistic images, which essentially inhibits the domain-shift noise in training
and optimization. Equipped with the elaborated GAR, we further proposed a novel
approach to predict 3D face parameters, in which we first obtain fine initial
parameters via Renderer Inverting and then refine it with gradient-based
optimizers. Extensive experiments have been conducted to demonstrate the
effectiveness of the proposed generative adversarial renderer and the novel
optimization-based face reconstruction framework. Our method achieves
state-of-the-art performances on multiple face reconstruction datasets.

本文提出了一种新型生成对抗渲染器（Generative Adversarial Renderer，简称 GAR）和用于预测 3D 面部参数的优化方法，该 GAR 利用神经渲染器而非传统的渲染规则来产生逼真图像，从而增强了训练和优化的稳定性，并在多个人脸重建数据集上取得了最新成果。