Most earlier investigations on talking face generation have focused on the
synchronization of lip motion and speech content. However, human head pose and
facial emotions are equally important characteristics of natural human faces.
While audio-driven talking face generation has seen notable advancements,
existing methods either overlook facial emotions or are limited to specific
individuals and cannot be applied to arbitrary subjects. In this paper, we
propose a one-shot Talking Head Generation framework (SPEAK) that distinguishes
itself from general Talking Face Generation by enabling emotional and postural
control. Specifically, we introduce the Inter-Reconstructed Feature
Disentanglement (IRFD) method to decouple human facial features into three
latent spaces. We then design a face editing module that modifies speech
content and facial latent codes into a single latent space. Subsequently, we
present a novel generator that employs modified latent codes derived from the
editing module to regulate emotional expression, head poses, and speech content
in synthesizing facial animations. Extensive trials demonstrate that our method
can generate realistic talking head with coordinated lip motions, authentic
facial emotions, and smooth head movements. The demo video is available at the
anonymous link: this https URL

提出了一种名为 SPEAK 的一次性 Talking Head Generation 框架，通过情感和姿势控制实现与一般 Talking Face Generation 的区别。该方法采用 Inter-Reconstructed Feature Disentanglement (IRFD) 方法将人脸特征解耦为三个潜在空间，并设计了一个面部编辑模块，将语音内容和面部潜在编码修改为单一的潜在空间。进一步，提出了一种新颖的生成器，利用编辑模块生成的修改后的潜在编码来调节情感表达、头部姿势和语音内容，以合成面部动画。大量实验表明，该方法可以生成具有协调的唇部运动、真实的面部情感和平滑的头部运动的逼真说话角色。