This paper proposes a talking face generation method named "CP-EB" that takes
an audio signal as input and a person image as reference, to synthesize a
photo-realistic people talking video with head poses controlled by a short
video clip and proper eye blinking embedding. It's noted that not only the head
pose but also eye blinking are both important aspects for deep fake detection.
The implicit control of poses by video has already achieved by the state-of-art
work. According to recent research, eye blinking has weak correlation with
input audio which means eye blinks extraction from audio and generation are
possible. Hence, we propose a GAN-based architecture to extract eye blink
feature from input audio and reference video respectively and employ
contrastive training between them, then embed it into the concatenated features
of identity and poses to generate talking face images. Experimental results
show that the proposed method can generate photo-realistic talking face with
synchronous lips motions, natural head poses and blinking eyes.

该研究提出了一种名为 'CP-EB' 的交谈面部生成方法，该方法以音频信号作为输入、以人物图像作为参考，通过控制一个短视频剪辑和适当的嵌入式眨眼操作，合成出一个由头部姿势控制的逼真人物对话视频。实验结果表明，该方法可以产生出具有同步嘴唇动作、自然头部姿势和眨眼效果的照片般逼真的交谈面部。