We propose a novel text-to-video (T2V) generation benchmark,
ChronoMagic-Bench, to evaluate the temporal and metamorphic capabilities of the
T2V models (e.g. Sora and Lumiere) in time-lapse video generation. In contrast
to existing benchmarks that focus on the visual quality and textual relevance
of generated videos, ChronoMagic-Bench focuses on the model's ability to
generate time-lapse videos with significant metamorphic amplitude and temporal
coherence. The benchmark probes T2V models for their physics, biology, and
chemistry capabilities, in a free-form text query. For these purposes,
ChronoMagic-Bench introduces 1,649 prompts and real-world videos as references,
categorized into four major types of time-lapse videos: biological,
human-created, meteorological, and physical phenomena, which are further
divided into 75 subcategories. This categorization comprehensively evaluates
the model's capacity to handle diverse and complex transformations. To
accurately align human preference with the benchmark, we introduce two new
automatic metrics, MTScore and CHScore, to evaluate the videos' metamorphic
attributes and temporal coherence. MTScore measures the metamorphic amplitude,
reflecting the degree of change over time, while CHScore assesses the temporal
coherence, ensuring the generated videos maintain logical progression and
continuity. Based on the ChronoMagic-Bench, we conduct comprehensive manual
evaluations of ten representative T2V models, revealing their strengths and
weaknesses across different categories of prompts, and providing a thorough
evaluation framework that addresses current gaps in video generation research.
Moreover, we create a large-scale ChronoMagic-Pro dataset, containing 460k
high-quality pairs of 720p time-lapse videos and detailed captions ensuring
high physical pertinence and large metamorphic amplitude.

我们提出了一个新颖的文本到视频（T2V）生成基准，ChronoMagic-Bench，用于评估 T2V 模型（例如 Sora 和 Lumiere）在延时摄影视频生成中的时间和变态能力。

ChronoMagic-Bench：用于文本转时间流影片生成的变形评估基准

ChronoMagic-Bench: A Benchmark for Metamorphic Evaluation of  Text-to-Time-lapse Video Generation

Taking a photo outside, can we predict the immediate future, e.g., how would
the cloud move in the sky? We address this problem by presenting a generative
adversarial network (GAN) based two-stage approach to generating realistic
time-lapse videos of high resolution. Given the first frame, our model learns
to generate long-term future frames. The first stage generates videos of
realistic contents for each frame. The second stage refines the generated video
from the first stage by enforcing it to be closer to real videos with regard to
motion dynamics. To further encourage vivid motion in the final generated
video, Gram matrix is employed to model the motion more precisely. We build a
large scale time-lapse dataset, and test our approach on this new dataset.
Using our model, we are able to generate realistic videos of up to $128\times
128$ resolution for 32 frames. Quantitative and qualitative experiment results
have demonstrated the superiority of our model over the state-of-the-art
models.

该研究基于 GAN，介绍了一种生成逼真的高分辨率时间流视频的方法，第一阶段生成真实内容的视频，第二阶段利用 Gram 矩阵提高了运动动态和最后生成视频的逼真性。实验证明，该方法优于现有的最新模型。

利用多阶段动态生成对抗网络学习生成时间变换视频

Learning to Generate Time-Lapse Videos Using Multi-Stage Dynamic  Generative Adversarial Networks

Based on life-long observations of physical, chemical, and biologic phenomena
in the natural world, humans can often easily picture in their minds what an
object will look like in the future. But, what about computers? In this paper,
we learn computational models of object transformations from time-lapse videos.
In particular, we explore the use of generative models to create depictions of
objects at future times. These models explore several different prediction
tasks: generating a future state given a single depiction of an object,
generating a future state given two depictions of an object at different times,
and generating future states recursively in a recurrent framework. We provide
both qualitative and quantitative evaluations of the generated results, and
also conduct a human evaluation to compare variations of our models.

本研究使用生成模型从时光缩影中计算得出物体变形，包括：生成单个物体的未来状态、生成两个不同时刻的物体状态以及在循环框架中递归生成未来状态，并通过定性和定量方法进行评估并与其他模型进行比较。