Recent advances in Text-to-Video generation (T2V) have achieved remarkable
success in synthesizing high-quality general videos from textual descriptions.
A largely overlooked problem in T2V is that existing models have not adequately
encoded physical knowledge of the real world, thus generated videos tend to
have limited motion and poor variations. In this paper, we propose
\textbf{MagicTime}, a metamorphic time-lapse video generation model, which
learns real-world physics knowledge from time-lapse videos and implements
metamorphic generation. First, we design a MagicAdapter scheme to decouple
spatial and temporal training, encode more physical knowledge from metamorphic
videos, and transform pre-trained T2V models to generate metamorphic videos.
Second, we introduce a Dynamic Frames Extraction strategy to adapt to
metamorphic time-lapse videos, which have a wider variation range and cover
dramatic object metamorphic processes, thus embodying more physical knowledge
than general videos. Finally, we introduce a Magic Text-Encoder to improve the
understanding of metamorphic video prompts. Furthermore, we create a time-lapse
video-text dataset called \textbf{ChronoMagic}, specifically curated to unlock
the metamorphic video generation ability. Extensive experiments demonstrate the
superiority and effectiveness of MagicTime for generating high-quality and
dynamic metamorphic videos, suggesting time-lapse video generation is a
promising path toward building metamorphic simulators of the physical world.

提出了 MagicTime，这是一个从时间间隔视频学习真实世界物理知识并实现变形生成的模型。设计了 MagicAdapter 方案来解耦空间和时间训练，引入了 Dynamic Frames Extraction 策略来适应变形时间间隔视频，并引入了 Magic Text-Encoder 来改善对变形视频提示的理解。通过实验证明了 MagicTime 在生成高质量和动态变形视频方面的优越性和有效性，表明时间间隔视频生成是构建物理世界变形模拟器的有希望的路径。