The advent of ChatGPT has introduced innovative methods for information
gathering and analysis. However, the information provided by ChatGPT is limited
to text, and the visualization of this information remains constrained.
Previous research has explored zero-shot text-to-video (TTV) approaches to
transform text into videos. However, these methods lacked control over the
identity of the generated audio, i.e., not identity-agnostic, hindering their
effectiveness. To address this limitation, we propose a novel two-stage
framework for person-agnostic video cloning, specifically focusing on TTV
generation. In the first stage, we leverage pretrained zero-shot models to
achieve text-to-speech (TTS) conversion. In the second stage, an audio-driven
talking head generation method is employed to produce compelling videos
privided the audio generated in the first stage. This paper presents a
comparative analysis of different TTS and audio-driven talking head generation
methods, identifying the most promising approach for future research and
development. Some audio and videos samples can be found in the following link:
this https URL

本文提出了一种新颖的两阶段框架，用于人物无关视频克隆，特别关注文本转视频生成。在第一阶段，我们利用预训练的零样本模型实现文本转语音转换。第二阶段采用音频驱动的说话人生成方法，根据第一阶段生成的音频产生引人注目的视频。该论文对不同的文本转语音和音频驱动的说话人生成方法进行了比较分析，确定了最有前景的研究和开发方法。