Given an arbitrary audio clip, audio-driven 3D facial animation aims to
generate lifelike lip motions and facial expressions for a 3D head. Existing
methods typically rely on training their models using limited public 3D
datasets that contain a restricted number of audio-3D scan pairs. Consequently,
their generalization capability remains limited. In this paper, we propose a
novel method that leverages in-the-wild 2D talking-head videos to train our 3D
facial animation model. The abundance of easily accessible 2D talking-head
videos equips our model with a robust generalization capability. By combining
these videos with existing 3D face reconstruction methods, our model excels in
generating consistent and high-fidelity lip synchronization. Additionally, our
model proficiently captures the speaking styles of different individuals,
allowing it to generate 3D talking-heads with distinct personal styles.
Extensive qualitative and quantitative experimental results demonstrate the
superiority of our method.

本文提出了一种基于 2D 口型视频和 3D 面部重构方法训练的 3D 面部动画模型，该模型具有良好的泛化能力和包括口型同步在内的高保真度，能够捕捉不同个体的讲话风格并输出个性化的 3D 说话头像，经实验证明其优越性。