Generating portrait images by controlling the motions of existing faces is an
important task of great consequence to social media industries. For easy use
and intuitive control, semantically meaningful and fully disentangled
parameters should be used as modifications. However, many existing techniques
do not provide such fine-grained controls or use indirect editing methods i.e.
mimic motions of other individuals. In this paper, a Portrait Image Neural
Renderer (PIRenderer) is proposed to control the face motions with the
parameters of three-dimensional morphable face models (3DMMs). The proposed
model can generate photo-realistic portrait images with accurate movements
according to intuitive modifications. Experiments on both direct and indirect
editing tasks demonstrate the superiority of this model. Meanwhile, we further
extend this model to tackle the audio-driven facial reenactment task by
extracting sequential motions from audio inputs. We show that our model can
generate coherent videos with convincing movements from only a single reference
image and a driving audio stream. Our source code is available at
this https URL

使用三维可变形面部模型参数来控制面部动作，生成逼真的肖像图像并实现音频驱动的面部再现。