We present an unsupervised approach for learning to estimate three
dimensional (3D) facial structure from a single image while also predicting 3D
viewpoint transformations that match a desired pose and facial geometry. We
achieve this by inferring the depth of facial keypoints of an input image in an
unsupervised manner, without using any form of ground-truth depth information.
We show how it is possible to use these depths as intermediate computations
within a new backpropable loss to predict the parameters of a 3D affine
transformation matrix that maps inferred 3D keypoints of an input face to the
corresponding 2D keypoints on a desired target facial geometry or pose. Our
resulting approach, called DepthNets, can therefore be used to infer plausible
3D transformations from one face pose to another, allowing faces to be
frontalized, transformed into 3D models or even warped to another pose and
facial geometry. Lastly, we identify certain shortcomings with our formulation,
and explore adversarial image translation techniques as a post-processing step
to re-synthesize complete head shots for faces re-targeted to different poses
or identities.

本文提出了一种无监督学习方法，通过推断面部关键点的深度来预测面部的三维结构和姿态变换，从而实现将面部从一个姿态、几何形状转换成另一个姿态、几何形状等多种功能；同时，探索了对该方法的不足之处，提出使用对抗图像翻译技术作为后处理步骤以重新合成不同姿态或身份的面部的完整头像。