Different people have different facial expressions while speaking
emotionally. A realistic facial animation system should consider such
identity-specific speaking styles and facial idiosyncrasies to achieve
high-degree of naturalness and plausibility. Existing approaches to
personalized speech-driven 3D facial animation either use one-hot identity
labels or rely-on person specific models which limit their scalability. We
present a personalized speech-driven expressive 3D facial animation synthesis
framework that models identity specific facial motion as latent representations
(called as styles), and synthesizes novel animations given a speech input with
the target style for various emotion categories. Our framework is trained in an
end-to-end fashion and has a non-autoregressive encoder-decoder architecture
with three main components: expression encoder, speech encoder and expression
decoder. Since, expressive facial motion includes both identity-specific style
and speech-related content information; expression encoder first disentangles
facial motion sequences into style and content representations, respectively.
Then, both of the speech encoder and the expression decoders input the
extracted style information to update transformer layer weights during training
phase. Our speech encoder also extracts speech phoneme label and duration
information to achieve better synchrony within the non-autoregressive synthesis
mechanism more effectively. Through detailed experiments, we demonstrate that
our approach produces temporally coherent facial expressions from input speech
while preserving the speaking styles of the target identities.

通过建模特定身份的面部动作为潜在表示（称为样式），并在各种情感类别的语音输入中合成具有目标样式的新动画，我们提出了一种个性化语音驱动的富有表现力的 3D 面部动画综合框架。