In this work we propose a novel model-based deep convolutional autoencoder that addresses the highly challenging problem of reconstructing a 3D human face from a single in-the-wild color image. To this end, we combine a convolutional encoder network with an expert-designed generative model that serves as decoder. The core innovation is our new differentiable parametric decoder that encapsulates image formation analytically based on a generative model. Our decoder takes as input a code vector with exactly defined semantic meaning that encodes detailed face pose, shape, expression, skin reflectance and scene illumination. Due to this new way of combining CNN-based with model-based face reconstruction, the CNN-based encoder learns to extract semantically meaningful parameters from a single monocular input image. For the first time, a CNN encoder and an expert-designed generative model can be trained end-to-end in an unsupervised manner, which renders training on very large (unlabeled) real world data feasible. The obtained reconstructions compare favorably to current state-of-the-art approaches in terms of quality and richness of representation.

本文提出了一种基于深度卷积自编码器的新颖模型，用于从野外拍摄的单幅彩色图像重建三维人脸。通过将卷积编码器与专家设计的生成模型相结合，提出了一种新的可微分参数解码器。该解码器将基于生成式模型的图像形成解析包括在内，以输入的码向量作为输入，从单个单眼输入图像中提取具有明确定义语义含义的参数。该文章的一个突破是在无监督的情况下，首次实现了CNN编码器和专家设计的生成模型的端到端训练，这使得对非常大的（未标记的）真实世界数据进行训练成为可能。所得到的重建结果在质量和表示的丰富性方面均优于当前最先进的方法。

MoFA：基于模型的深度卷积自编码器用于单目无监督重建