The goal of this work is to simultaneously generate natural talking faces and
speech outputs from text. We achieve this by integrating Talking Face
Generation (TFG) and Text-to-Speech (TTS) systems into a unified framework. We
address the main challenges of each task: (1) generating a range of head poses
representative of real-world scenarios, and (2) ensuring voice consistency
despite variations in facial motion for the same identity. To tackle these
issues, we introduce a motion sampler based on conditional flow matching, which
is capable of high-quality motion code generation in an efficient way.
Moreover, we introduce a novel conditioning method for the TTS system, which
utilises motion-removed features from the TFG model to yield uniform speech
outputs. Our extensive experiments demonstrate that our method effectively
creates natural-looking talking faces and speech that accurately match the
input text. To our knowledge, this is the first effort to build a multimodal
synthesis system that can generalise to unseen identities.

利用文本生成自然的说话面部表情和语音输出，通过将 Talking Face Generation (TFG) 和 Text-to-Speech (TTS) 系统整合到一个统一框架中，在解决头部姿态生成和声音一致性等挑战的同时，实现了高质量的运动代码生成和均匀语音输出。