The goal of this paper is to synthesise talking faces with controllable
facial motions. To achieve this goal, we propose two key ideas. The first is to
establish a canonical space where every face has the same motion patterns but
different identities. The second is to navigate a multimodal motion space that
only represents motion-related features while eliminating identity information.
To disentangle identity and motion, we introduce an orthogonality constraint
between the two different latent spaces. From this, our method can generate
natural-looking talking faces with fully controllable facial attributes and
accurate lip synchronisation. Extensive experiments demonstrate that our method
achieves state-of-the-art results in terms of both visual quality and lip-sync
score. To the best of our knowledge, we are the first to develop a talking face
generation framework that can accurately manifest full target facial motions
including lip, head pose, and eye movements in the generated video without any
additional supervision beyond RGB video with audio.

论文旨在综合具有可控面部动作的谈话面孔。通过建立规范空间和多模态运动空间，提出正交性约束来分离身份和运动，从而实现生成外观自然、具有完全可控面部属性和准确的唇部同步的谈话面孔。实验证明，我们的方法在视觉质量和唇同步得分方面均取得了最先进的结果。这是第一个开发出在生成的视频中准确地展示包括唇部，头部姿势和眼动等所有目标面部运动的谈话面孔生成框架，没有超越 RGB 视频与音频的任何附加监督。