Talking face generation has gained immense popularity in the computer vision
community, with various applications including AR/VR, teleconferencing, digital
assistants, and avatars. Traditional methods are mainly audio-driven ones which
have to deal with the inevitable resource-intensive nature of audio storage and
processing. To address such a challenge, we propose FT2TF - First-Person
Statement Text-To-Talking Face Generation, a novel one-stage end-to-end
pipeline for talking face generation driven by first-person statement text.
Moreover, FT2TF implements accurate manipulation of the facial expressions by
altering the corresponding input text. Different from previous work, our model
only leverages visual and textual information without any other sources (e.g.
audio/landmark/pose) during inference. Extensive experiments are conducted on
LRS2 and LRS3 datasets, and results on multi-dimensional evaluation metrics are
reported. Both quantitative and qualitative results showcase that FT2TF
outperforms existing relevant methods and reaches the state-of-the-art. This
achievement highlights our model capability to bridge first-person statements
and dynamic face generation, providing insightful guidance for future work.

提出了 FT2TF - 第一人称陈述文本到说话人脸生成的新的一阶段端到端流水线，通过改变相应的输入文本实现对面部表情的准确操作，证明在多维度评估指标上优于现有相关方法并达到最先进水平。