This paper introduces ModelScopeT2V, a text-to-video synthesis model that
evolves from a text-to-image synthesis model (i.e., Stable Diffusion).
ModelScopeT2V incorporates spatio-temporal blocks to ensure consistent frame
generation and smooth movement transitions. The model could adapt to varying
frame numbers during training and inference, rendering it suitable for both
image-text and video-text datasets. ModelScopeT2V brings together three
components (i.e., VQGAN, a text encoder, and a denoising UNet), totally
comprising 1.7 billion parameters, in which 0.5 billion parameters are
dedicated to temporal capabilities. The model demonstrates superior performance
over state-of-the-art methods across three evaluation metrics. The code and an
online demo are available at
https://modelscope.cn/models/damo/text-to-video-synthesis/summary.

ModelScopeT2V 是一个从文本到图像合成模型演变而来的文本到视频合成模型，它包含时空块以确保一致的帧生成和平滑的运动过渡。该模型可以适应不同的帧数量，在训练和推断过程中均适用于图像 - 文本和视频 - 文本数据集。拥有 17 亿参数的 ModelScopeT2V，整体由 VQGAN、文本编码器和去噪 UNet 组成，其中有 5 亿参数专门用于时序能力。该模型在三个评估指标上表现出优越的性能，优于现有的方法。代码和在线演示详见 https://modelscope.cn/models/damo/text-to-video-synthesis/summary。