We present a novel face reconstruction method capable of reconstructing
detailed face geometry, spatially varying face reflectance from a single
monocular image. We build our work upon the recent advances of DNN-based
auto-encoders with differentiable ray tracing image formation, trained in
self-supervised manner. While providing the advantage of learning-based
approaches and real-time reconstruction, the latter methods lacked fidelity. In
this work, we achieve, for the first time, high fidelity face reconstruction
using self-supervised learning only. Our novel coarse-to-fine deep architecture
allows us to solve the challenging problem of decoupling face reflectance from
geometry using a single image, at high computational speed. Compared to
state-of-the-art methods, our method achieves more visually appealing
reconstruction.

本文提出一种能够重建面部几何形状、在单个单目图像中重建时空变化的面部反射的新颖面部重建方法，采用基于 DNN 的自编码器和可微分光线追踪图像构建，同时利用自监督训练方法实现高保真度的面部重建，同时实现了面部反射和几何形状的高度解耦。

S2F2: 自监督单目图像高保真度人脸重建

S2F2: Self-Supervised High Fidelity Face Reconstruction from Monocular  Image

The reflectance field of a face describes the reflectance properties
responsible for complex lighting effects including diffuse, specular,
inter-reflection and self shadowing. Most existing methods for estimating the
face reflectance from a monocular image assume faces to be diffuse with very
few approaches adding a specular component. This still leaves out important
perceptual aspects of reflectance as higher-order global illumination effects
and self-shadowing are not modeled. We present a new neural representation for
face reflectance where we can estimate all components of the reflectance
responsible for the final appearance from a single monocular image. Instead of
modeling each component of the reflectance separately using parametric models,
our neural representation allows us to generate a basis set of faces in a
geometric deformation-invariant space, parameterized by the input light
direction, viewpoint and face geometry. We learn to reconstruct this
reflectance field of a face just from a monocular image, which can be used to
render the face from any viewpoint in any light condition. Our method is
trained on a light-stage training dataset, which captures 300 people
illuminated with 150 light conditions from 8 viewpoints. We show that our
method outperforms existing monocular reflectance reconstruction methods, in
terms of photorealism due to better capturing of physical premitives, such as
sub-surface scattering, specularities, self-shadows and other higher-order
effects.

本研究提出一种基于神经表示的脸部反射模型，通过单张图像的估计来生成基于光照、视角和脸部几何变换的脸部反射基，可用于在任何光照条件下从任何视角渲染脸部，其能更好地捕捉诸如亚表面散射、镜面效应、自身阴影和其他高阶效应等物理基元，进而提高光照模型的仿真度。