We present a minimalistic but effective neural network that computes dense
facial correspondences in highly unconstrained RGB images. Our network learns a
per-pixel flow and a matchability mask between 2D input photographs of a person
and the projection of a textured 3D face model. To train such a network, we
generate a massive dataset of synthetic faces with dense labels using
renderings of a morphable face model with variations in pose, expressions,
lighting, and occlusions. We found that a training refinement using real
photographs is required to drastically improve the ability to handle real
images. When combined with a facial detection and 3D face fitting step, we show
that our approach outperforms the state-of-the-art face alignment methods in
terms of accuracy and speed. By directly estimating dense correspondences, we
do not rely on the full visibility of sparse facial landmarks and are not
limited to the model space of regression-based approaches. We also assess our
method on video frames and demonstrate successful per-frame processing under
extreme pose variations, occlusions, and lighting conditions. Compared to
existing 3D facial tracking techniques, our fitting does not rely on previous
frames or frontal facial initialization and is robust to imperfect face
detections.

本文提出了一种使用神经网络在高度自由的 RGB 图像中计算面部对应关系的最简单有效的方法，使用渲染可变姿势、表情、照明和遮挡的多样化三维面部模型生成大量人工数据集，结合真实照片培训表现在精度和速度上优于现有方法，并可以成功地处理极端姿势变化、遮挡和光照条件。