We present 3DiFACE, a novel method for personalized speech-driven 3D facial
animation and editing. While existing methods deterministically predict facial
animations from speech, they overlook the inherent one-to-many relationship
between speech and facial expressions, i.e., there are multiple reasonable
facial expression animations matching an audio input. It is especially
important in content creation to be able to modify generated motion or to
specify keyframes. To enable stochasticity as well as motion editing, we
propose a lightweight audio-conditioned diffusion model for 3D facial motion.
This diffusion model can be trained on a small 3D motion dataset, maintaining
expressive lip motion output. In addition, it can be finetuned for specific
subjects, requiring only a short video of the person. Through quantitative and
qualitative evaluations, we show that our method outperforms existing
state-of-the-art techniques and yields speech-driven animations with greater
fidelity and diversity.

我们提出的 3DiFACE 方法是一种用于个性化语音驱动的 3D 面部动画和编辑的新方法。通过引入一种轻量级的音频条件扩散模型，我们能够在保持表现力丰富的唇部运动输出的同时，允许随机性和动作编辑。经过定量和定性评估，我们展示了我们的方法优于现有的技术，并产生更具保真度和多样性的语音驱动动画。