Performing facial expression transfer under one-shot setting has been
increasing in popularity among research community with a focus on precise
control of expressions. Existing techniques showcase compelling results in
perceiving expressions, but they lack robustness with extreme head poses. They
also struggle to accurately reconstruct background details, thus hindering the
realism. In this paper, we propose a novel warping technology which integrates
the advantages of both 2D and 3D methods to achieve robust face re-enactment.
We generate dense 3D facial flow fields in feature space to warp an input image
based on target expressions without depth information. This enables explicit 3D
geometric control for re-enacting misaligned source and target faces. We
regularize the motion estimation capability of the 3D flow prediction network
through proposed "Cyclic warp loss" by converting warped 3D features back into
2D RGB space. To ensure the generation of finer facial region with
natural-background, our framework only renders the facial foreground region
first and learns to inpaint the blank area which needs to be filled due to
source face translation, thus reconstructing the detailed background without
any unwanted pixel motion. Extensive evaluation reveals that our method
outperforms state-of-the-art techniques in rendering artifact-free facial
images.

在这篇论文中，我们提出了一种新的变形技术，将 2D 和 3D 方法的优势结合起来，以实现稳健的面部再演技术。我们生成了密集的 3D 面部流场，并通过所提出的 “循环变形损失” 将其转换回 2D RGB 空间，从而实现了对源面孔和目标面孔的 3D 几何控制。通过首先渲染面部前景区域，然后学习对由于源面孔平移而需要填充的空白区域进行修复，我们重建了细节丰富的背景，而且在渲染无伪影的面部图像方面超过了现有技术。