Creating realistic or stylized facial and lip sync animation is a tedious
task. It requires lot of time and skills to sync the lips with audio and convey
the right emotion to the character's face. To allow animators to spend more
time on the artistic and creative part of the animation, we present Audio2Rig:
a new deep learning based tool leveraging previously animated sequences of a
show, to generate facial and lip sync rig animation from an audio file. Based
in Maya, it learns from any production rig without any adjustment and generates
high quality and stylized animations which mimic the style of the show.
Audio2Rig fits in the animator workflow: since it generates keys on the rig
controllers, the animation can be easily retaken. The method is based on 3
neural network modules which can learn an arbitrary number of controllers.
Hence, different configurations can be created for specific parts of the face
(such as the tongue, lips or eyes). With Audio2Rig, animators can also pick
different emotions and adjust their intensities to experiment or customize the
output, and have high level controls on the keyframes setting. Our method shows
excellent results, generating fine animation details while respecting the show
style. Finally, as the training relies on the studio data and is done
internally, it ensures data privacy and prevents from copyright infringement.

Audio2Rig 是一种基于深度学习的工具，通过从音频文件中生成面部和嘴唇同步的动画来减少动画师所需的时间和技能，同时保持高质量和风格一致性。