Most 3D face reconstruction methods rely on 3D morphable models, which
disentangle the space of facial deformations into identity geometry,
expressions and skin reflectance. These models are typically learned from a
limited number of 3D scans and thus do not generalize well across different
identities and expressions. We present the first approach to learn complete 3D
models of face identity geometry, albedo and expression just from images and
videos. The virtually endless collection of such data, in combination with our
self-supervised learning-based approach allows for learning face models that
generalize beyond the span of existing approaches. Our network design and loss
functions ensure a disentangled parameterization of not only identity and
albedo, but also, for the first time, an expression basis. Our method also
allows for in-the-wild monocular reconstruction at test time. We show that our
learned models better generalize and lead to higher quality image-based
reconstructions than existing approaches.

本研究提出了一种基于自监督学习的方法，通过图像和视频数据学习脸部特征的完整 3D 模型，实现了环境无关的单目重建，并训练出能够更好地推广和提高图像重建质量的表情基础等方面的面部模型。

从图像和视频中学习完整的三维可塑性人脸模型

Learning Complete 3D Morphable Face Models from Images and Videos

Given the recent advances in depth prediction from Convolutional Neural
Networks (CNNs), this paper investigates how predicted depth maps from a deep
neural network can be deployed for accurate and dense monocular reconstruction.
We propose a method where CNN-predicted dense depth maps are naturally fused
together with depth measurements obtained from direct monocular SLAM. Our
fusion scheme privileges depth prediction in image locations where monocular
SLAM approaches tend to fail, e.g. along low-textured regions, and vice-versa.
We demonstrate the use of depth prediction for estimating the absolute scale of
the reconstruction, hence overcoming one of the major limitations of monocular
SLAM. Finally, we propose a framework to efficiently fuse semantic labels,
obtained from a single frame, with dense SLAM, yielding semantically coherent
scene reconstruction from a single view. Evaluation results on two benchmark
datasets show the robustness and accuracy of our approach.

本文研究了如何利用卷积神经网络深度预测图来进行准确而密集的单目重建，并提出了一种将 CNN 预测的密集深度图与直接单目 SLAM 测量所获得的深度测量自然融合的方法，证明了这种方法在场景重建绝对尺度估计等方面的鲁棒性和准确性。最终，提出了一个将语义标签与单个帧的密集 SLAM 高效融合的框架，能够从单个视图中获得语义一致的场景重建。