Generating emotional talking faces is a practical yet challenging endeavor.
To create a lifelike avatar, we draw upon two critical insights from a human
perspective: 1) The connection between audio and the non-deterministic facial
dynamics, encompassing expressions, blinks, poses, should exhibit synchronous
and one-to-many mapping. 2) Vibrant expressions are often accompanied by
emotion-aware high-definition (HD) textures and finely detailed teeth. However,
both aspects are frequently overlooked by existing methods. To this end, this
paper proposes using normalizing Flow and Vector-Quantization modeling to
produce emotional talking faces that satisfy both insights concurrently
(FlowVQTalker). Specifically, we develop a flow-based coefficient generator
that encodes the dynamics of facial emotion into a multi-emotion-class latent
space represented as a mixture distribution. The generation process commences
with random sampling from the modeled distribution, guided by the accompanying
audio, enabling both lip-synchronization and the uncertain nonverbal facial
cues generation. Furthermore, our designed vector-quantization image generator
treats the creation of expressive facial images as a code query task, utilizing
a learned codebook to provide rich, high-quality textures that enhance the
emotional perception of the results. Extensive experiments are conducted to
showcase the effectiveness of our approach.

利用归一化流和向量量化建模生成既满足面部动态的同步性和多对多映射关系，又包含高清纹理和细节牙齿的情感说话脸，具体方法为通过流式系数生成器对面部情绪进行编码，利用伴随音频引导随机采样生成唇同步和不确定的非言语面部线索，并借助学习的码本提供富有表现力、高质量的纹理，从而增强结果的情感感知。