We present a novel deep-learning based approach to producing animator-centric
speech motion curves that drive a JALI or standard FACS-based production
face-rig, directly from input audio. Our three-stage Long Short-Term Memory
(LSTM) network architecture is motivated by psycho-linguistic insights:
segmenting speech audio into a stream of phonetic-groups is sufficient for
viseme construction; speech styles like mumbling or shouting are strongly
co-related to the motion of facial landmarks; and animator style is encoded in
viseme motion curve profiles. Our contribution is an automatic real-time
lip-synchronization from audio solution that integrates seamlessly into
existing animation pipelines. We evaluate our results by: cross-validation to
ground-truth data; animator critique and edits; visual comparison to recent
deep-learning lip-synchronization solutions; and showing our approach to be
resilient to diversity in speaker and language.

本文提出了一种基于深度学习的新型方法，用于直接从输入音频中产生驱动 JALI 或标准 FACS 制作面部捕捉的动画师中心音频运动曲线。