Recent works have shown how realistic talking face images can be obtained
under the supervision of geometry guidance, e.g., facial landmark or boundary.
To alleviate the demand for manual annotations, in this paper, we propose a
novel self-supervised hybrid model (DAE-GAN) that learns how to reenact face
naturally given large amounts of unlabeled videos. Our approach combines two
deforming autoencoders with the latest advances in the conditional generation.
On the one hand, we adopt the deforming autoencoder to disentangle identity and
pose representations. A strong prior in talking face videos is that each frame
can be encoded as two parts: one for video-specific identity and the other for
various poses. Inspired by that, we utilize a multi-frame deforming autoencoder
to learn a pose-invariant embedded face for each video. Meanwhile, a
multi-scale deforming autoencoder is proposed to extract pose-related
information for each frame. On the other hand, the conditional generator allows
for enhancing fine details and overall reality. It leverages the disentangled
features to generate photo-realistic and pose-alike face images. We evaluate
our model on VoxCeleb1 and RaFD dataset. Experiment results demonstrate the
superior quality of reenacted images and the flexibility of transferring facial
movements between identities.

本文提出了一种新型的自我监督混合模型（DAE-GAN），它结合了两个形变自编码器及条件生成的最新进展，用于学习如何在大量未标记视频的情况下自然地再现人脸，并且在 VoxCeleb1 和 RaFD 数据集上得到了优秀的实验结果，表明了重新表演图像的优异品质和在不同身份之间转移面部动作的灵活性。

通过自监督分离身份和姿势实现逼真的人脸复原

Realistic Face Reenactment via Self-Supervised Disentangling of Identity  and Pose

In this work we introduce Deforming Autoencoders, a generative model for
images that disentangles shape from appearance in an unsupervised manner. As in
the deformable template paradigm, shape is represented as a deformation between
a canonical coordinate system (`template') and an observed image, while
appearance is modeled in `canonical', template, coordinates, thus discarding
variability due to deformations. We introduce novel techniques that allow this
approach to be deployed in the setting of autoencoders and show that this
method can be used for unsupervised group-wise image alignment. We show
experiments with expression morphing in humans, hands, and digits, face
manipulation, such as shape and appearance interpolation, as well as
unsupervised landmark localization. A more powerful form of unsupervised
disentangling becomes possible in template coordinates, allowing us to
successfully decompose face images into shading and albedo, and further
manipulate face images.

Deforming Autoencoders 是一种用于图像生成的生成模型，通过无监督方式将形状与外观分离。使用模板坐标系表示形状，并且可以成功分解面部图像成明暗和反照率两部分，在模板坐标系上更强大的无监督分解变得可能。