Video retargeting for digital face animation is used in virtual reality,
social media, gaming, movies, and video conference, aiming to animate avatars'
facial expressions based on videos of human faces. The standard method to
represent facial expressions for 3D characters is by blendshapes, a vector of
weights representing the avatar's neutral shape and its variations under facial
expressions, e.g., smile, puff, blinking. Datasets of paired frames with
blendshape vectors are rare, and labeling can be laborious, time-consuming, and
subjective. In this work, we developed an approach that handles the lack of
appropriate datasets. Instead, we used a synthetic dataset of only one
character. To generalize various characters, we re-represented each frame to
face landmarks. We developed a unique deep-learning architecture that groups
landmarks for each facial organ and connects them to relevant blendshape
weights. Additionally, we incorporated complementary methods for facial
expressions that landmarks did not represent well and gave special attention to
eye expressions. We have demonstrated the superiority of our approach to
previous research in qualitative and quantitative metrics. Our approach
achieved a higher MOS of 68% and a lower MSE of 44.2% when tested on videos
with various users and expressions.

本文提出了一种使用合成数据集和重新表示技术来处理虚拟人物面部表情的深度学习框架，同时结合了针对面部器官的组群和关键点与权重相连的方法来改进以往的研究，同时在定量和定性指标上更优。