In this paper, we introduce a novel deep learning method for photo-realistic
manipulation of the emotional state of actors in "in-the-wild" videos. The
proposed method is based on a parametric 3D face representation of the actor in
the input scene that offers a reliable disentanglement of the facial identity
from the head pose and facial expressions. It then uses a novel deep domain
translation framework that alters the facial expressions in a consistent and
plausible manner, taking into account their dynamics. Finally, the altered
facial expressions are used to photo-realistically manipulate the facial region
in the input scene based on an especially-designed neural face renderer. To the
best of our knowledge, our method is the first to be capable of controlling the
actor's facial expressions by even using as a sole input the semantic labels of
the manipulated emotions, while at the same time preserving the speech-related
lip movements. We conduct extensive qualitative and quantitative evaluations
and comparisons, which demonstrate the effectiveness of our approach and the
especially promising results that we obtain. Our method opens a plethora of new
possibilities for useful applications of neural rendering technologies, ranging
from movie post-production and video games to photo-realistic affective
avatars.

本文介绍了一种新颖的深度学习方法，用于对 “野外” 视频中演员情绪状态的逼真操作。该方法基于演员输入场景中的参数化 3D 面部表示，并使用新颖的深度域转换框架，结合动态信息，以一致而可信的方式改变面部表情。经过广泛的定性和定量评估和比较，我们的方法证明了其有效性并取得了尤为有前途的结果。该方法可应用于电影后期制作、视频游戏和逼真的情感化头像等各种领域。