This paper introduces a new model to generate rhythmically relevant non-verbal facial behaviors for virtual agents while they speak. The model demonstrates perceived performance comparable to behaviors directly extracted from the data and replayed on a virtual agent, in terms of synchronization with speech and believability. Interestingly, we found that training the model with two different sets of data, instead of one, did not necessarily improve its performance. The expressiveness of the people in the dataset and the shooting conditions are key elements. We also show that employing an adversarial model, in which fabricated fake examples are introduced during the training phase, increases the perception of synchronization with speech. A collection of videos demonstrating the results and code can be accessed at: https://github.com/aldelb/non_verbal_facial_animation.

介绍了一种用于生成与语音同步、可信度较高的虚拟代理人非语言面部行为的新模型，并发现使用两个不同的数据集而不是一个并不能必然提高模型的性能。同时表明在训练阶段引入虚假的样本可以增加与语音的同步感。

朝向一个说话虚拟代理的同步且可信的非语言面部行为生成