Implementing fine-grained emotion control is crucial for emotion generation
tasks because it enhances the expressive capability of the generative model,
allowing it to accurately and comprehensively capture and express various
nuanced emotional states, thereby improving the emotional quality and
personalization of generated content. Generating fine-grained facial animations
that accurately portray emotional expressions using only a portrait and an
audio recording presents a challenge. In order to address this challenge, we
propose a visual attribute-guided audio decoupler. This enables the obtention
of content vectors solely related to the audio content, enhancing the stability
of subsequent lip movement coefficient predictions. To achieve more precise
emotional expression, we introduce a fine-grained emotion coefficient
prediction module. Additionally, we propose an emotion intensity control method
using a fine-grained emotion matrix. Through these, effective control over
emotional expression in the generated videos and finer classification of
emotion intensity are accomplished. Subsequently, a series of 3DMM coefficient
generation networks are designed to predict 3D coefficients, followed by the
utilization of a rendering network to generate the final video. Our
experimental results demonstrate that our proposed method, EmoSpeaker,
outperforms existing emotional talking face generation methods in terms of
expression variation and lip synchronization. Project page:
this https URL

通过引入细粒度情绪控制和音频分离模块来实现精确表达细致情绪状态的面部动画生成，有效控制情感表达并提高生成视频的表达变化和嘴唇同步性能。