Despite previous success in generating audio-driven talking heads, most of
the previous studies focus on the correlation between speech content and the
mouth shape. Facial emotion, which is one of the most important features on
natural human faces, is always neglected in their methods. In this work, we
present Emotional Video Portraits (EVP), a system for synthesizing high-quality
video portraits with vivid emotional dynamics driven by audios. Specifically,
we propose the Cross-Reconstructed Emotion Disentanglement technique to
decompose speech into two decoupled spaces, i.e., a duration-independent
emotion space and a duration dependent content space. With the disentangled
features, dynamic 2D emotional facial landmarks can be deduced. Then we propose
the Target-Adaptive Face Synthesis technique to generate the final high-quality
video portraits, by bridging the gap between the deduced landmarks and the
natural head poses of target videos. Extensive experiments demonstrate the
effectiveness of our method both qualitatively and quantitatively.

本文提出一种称为情感视频肖像 (EVP) 的系统，通过 Cross-Reconstructed Emotion Disentanglement 技术来将语音分解为情感和内容空间并提取二维情感面部特征，然后通过 Target-Adaptive Face Synthesis 技术来生成高质量的情感动态视频人像，该方法在定性和定量实验中均表现出良好的效果。