Reenacting facial images is an important task that can find numerous
applications. We proposed IFaceUV, a fully differentiable pipeline that
properly combines 2D and 3D information to conduct the facial reenactment task.
The three-dimensional morphable face models (3DMMs) and corresponding UV maps
are utilized to intuitively control facial motions and textures, respectively.
Two-dimensional techniques based on 2D image warping is further required to
compensate for missing components of the 3DMMs such as backgrounds, ear, hair
and etc. In our pipeline, we first extract 3DMM parameters and corresponding UV
maps from source and target images. Then, initial UV maps are refined by the UV
map refinement network and it is rendered to the image with the motion
manipulated 3DMM parameters. In parallel, we warp the source image according to
the 2D flow field obtained from the 2D warping network. Rendered and warped
images are combined in the final editing network to generate the final
reenactment image. Additionally, we tested our model for the audio-driven
facial reenactment task. Extensive qualitative and quantitative experiments
illustrate the remarkable performance of our method compared to other
state-of-the-art methods.

IFaceUV 提出了一种全可微的技术流程，该流程涵盖了 2D 和 3D 信息的所有关键方面，从而为面部再现任务提供了一个效果显着的解决方案。

IFaceUV：通过 UV 映射保留身份特征的直观动态面部图像生成

IFaceUV: Intuitive Motion Facial Image Generation by Identity  Preservation via UV map

Generating portrait images by controlling the motions of existing faces is an
important task of great consequence to social media industries. For easy use
and intuitive control, semantically meaningful and fully disentangled
parameters should be used as modifications. However, many existing techniques
do not provide such fine-grained controls or use indirect editing methods i.e.
mimic motions of other individuals. In this paper, a Portrait Image Neural
Renderer (PIRenderer) is proposed to control the face motions with the
parameters of three-dimensional morphable face models (3DMMs). The proposed
model can generate photo-realistic portrait images with accurate movements
according to intuitive modifications. Experiments on both direct and indirect
editing tasks demonstrate the superiority of this model. Meanwhile, we further
extend this model to tackle the audio-driven facial reenactment task by
extracting sequential motions from audio inputs. We show that our model can
generate coherent videos with convincing movements from only a single reference
image and a driving audio stream. Our source code is available at
this https URL

使用三维可变形面部模型参数来控制面部动作，生成逼真的肖像图像并实现音频驱动的面部再现。