We address the problem of recovering the 3D geometry of a human face from a
set of facial images in multiple views. While recent studies have shown
impressive progress in 3D Morphable Model (3DMM) based facial reconstruction,
the settings are mostly restricted to a single view. There is an inherent
drawback in the single-view setting: the lack of reliable 3D constraints can
cause unresolvable ambiguities. We in this paper explore 3DMM-based shape
recovery in a different setting, where a set of multi-view facial images are
given as input. A novel approach is proposed to regress 3DMM parameters from
multi-view inputs with an end-to-end trainable Convolutional Neural Network
(CNN). Multiview geometric constraints are incorporated into the network by
establishing dense correspondences between different views leveraging a novel
self-supervised view alignment loss. The main ingredient of the view alignment
loss is a differentiable dense optical flow estimator that can backpropagate
the alignment errors between an input view and a synthetic rendering from
another input view, which is projected to the target view through the 3D shape
to be inferred. Through minimizing the view alignment loss, better 3D shapes
can be recovered such that the synthetic projections from one view to another
can better align with the observed image. Extensive experiments demonstrate the
superiority of the proposed method over other 3DMM methods.

本文提出一种基于卷积神经网络和多视图图像的 3D 人脸重建方法，使用自监督的视角对齐损失来减少视图之间对齐误差，并使用光流法预测 3D 形状，实现更好的 3D 重建结果。

MVF-Net: 多视角 3D 人脸可塑模型回归

MVF-Net: Multi-View 3D Face Morphable Model Regression

We propose a straightforward method that simultaneously reconstructs the 3D
facial structure and provides dense alignment. To achieve this, we design a 2D
representation called UV position map which records the 3D shape of a complete
face in UV space, then train a simple Convolutional Neural Network to regress
it from a single 2D image. We also integrate a weight mask into the loss
function during training to improve the performance of the network. Our method
does not rely on any prior face model, and can reconstruct full facial geometry
along with semantic meaning. Meanwhile, our network is very light-weighted and
spends only 9.8ms to process an image, which is extremely faster than previous
works. Experiments on multiple challenging datasets show that our method
surpasses other state-of-the-art methods on both reconstruction and alignment
tasks by a large margin.

本文提出一种简单的方法，可以同时重建三维人脸结构并提供密集对齐。该方法使用称为 UV 位置映射的二维表示来记录完整面部的三维形状，并使用简单卷积神经网络从单个二维图像中回归它。该方法不依赖于任何先前的人脸模型，并且可以重建全面部几何信息。与之前的作品相比，网络非常轻便，并且只需 9.8 毫秒即可处理图像。多次挑战数据集的实验表明，该方法在重建和对齐任务上均优于其他最先进的方法。