Speech-driven 3D facial animation aims at generating facial movements that
are synchronized with the driving speech, which has been widely explored
recently. Existing works mostly neglect the person-specific talking style in
generation, including facial expression and head pose styles. Several works
intend to capture the personalities by fine-tuning modules. However, limited
training data leads to the lack of vividness. In this work, we propose AdaMesh,
a novel adaptive speech-driven facial animation approach, which learns the
personalized talking style from a reference video of about 10 seconds and
generates vivid facial expressions and head poses. Specifically, we propose
mixture-of-low-rank adaptation (MoLoRA) to fine-tune the expression adapter,
which efficiently captures the facial expression style. For the personalized
pose style, we propose a pose adapter by building a discrete pose prior and
retrieving the appropriate style embedding with a semantic-aware pose style
matrix without fine-tuning. Extensive experimental results show that our
approach outperforms state-of-the-art methods, preserves the talking style in
the reference video, and generates vivid facial animation. The supplementary
video and code will be available at this https URL

通过 AdaMesh 方法，我们能够从仅有大约 10 秒的参考视频中学习个性化的说话风格，生成生动的面部表情和头部姿势，超过了现有方法并保留了参考视频中的说话风格。