Talking face generation has been extensively investigated owing to its wide
applicability. The two primary frameworks used for talking face generation
comprise a text-driven framework, which generates synchronized speech and
talking faces from text, and a speech-driven framework, which generates talking
faces from speech. To integrate these frameworks, this paper proposes a unified
facial landmark generator (UniFLG). The proposed system exploits end-to-end
text-to-speech not only for synthesizing speech but also for extracting a
series of latent representations that are common to text and speech, and feeds
it to a landmark decoder to generate facial landmarks. We demonstrate that our
system achieves higher naturalness in both speech synthesis and facial landmark
generation compared to the state-of-the-art text-driven method. We further
demonstrate that our system can generate facial landmarks from speech of
speakers without facial video data or even speech data.

提出一种统一的面部标记生成器（Unified facial landmark generator），利用端到端的文本到语音不仅用于合成语音，还用于提取一系列与文本和语音共同的潜在表示，将其馈送至标记解码器以生成面部标记，并演示这个系统在语音合成和面部地标生成方面比最先进的基于文本的方法具有更高的真实性，可以从没有面部视频数据甚至语音数据的演讲者的语音中生成面部特征点。