Video diffusion models has been gaining increasing attention for its ability
to produce videos that are both coherent and of high fidelity. However, the
iterative denoising process makes it computationally intensive and
time-consuming, thus limiting its applications. Inspired by the Consistency
Model (CM) that distills pretrained image diffusion models to accelerate the
sampling with minimal steps and its successful extension Latent Consistency
Model (LCM) on conditional image generation, we propose AnimateLCM, allowing
for high-fidelity video generation within minimal steps. Instead of directly
conducting consistency learning on the raw video dataset, we propose a
decoupled consistency learning strategy that decouples the distillation of
image generation priors and motion generation priors, which improves the
training efficiency and enhance the generation visual quality. Additionally, to
enable the combination of plug-and-play adapters in stable diffusion community
to achieve various functions (e.g., ControlNet for controllable generation). we
propose an efficient strategy to adapt existing adapters to our distilled
text-conditioned video consistency model or train adapters from scratch without
harming the sampling speed. We validate the proposed strategy in
image-conditioned video generation and layout-conditioned video generation, all
achieving top-performing results. Experimental results validate the
effectiveness of our proposed method. Code and weights will be made public.
More details are available at this https URL

通过提出的 AnimaLCM 方法，我们可以在最小的步骤内实现高保真度的视频生成，通过解耦图像生成先验和动作生成先验的一致性学习策略，提高了训练效率和生成的视觉质量。同时，我们还提出了一种有效的策略，使得现有的适配器可在稳定的扩散社区内实现各种功能，同时不影响采样速度，验证实验结果表明了我们提出方法的有效性。

AnimateLCM：个性化扩散模型和适配器的动画加速与解耦一致性学习

AnimateLCM: Accelerating the Animation of Personalized Diffusion Models  and Adapters with Decoupled Consistency Learning

The growing demand for high-fidelity video generation from textual
descriptions has catalyzed significant research in this field. In this work, we
introduce MagicVideo-V2 that integrates the text-to-image model, video motion
generator, reference image embedding module and frame interpolation module into
an end-to-end video generation pipeline. Benefiting from these architecture
designs, MagicVideo-V2 can generate an aesthetically pleasing, high-resolution
video with remarkable fidelity and smoothness. It demonstrates superior
performance over leading Text-to-Video systems such as Runway, Pika 1.0, Morph,
Moon Valley and Stable Video Diffusion model via user evaluation at large
scale.

MagicVideo-V2 通过将文本到图像模型、视频运动生成器、参考图像嵌入模块和帧插值模块整合到端到端视频生成流程中，可以生成具有出色保真度和平滑度的美观、高分辨率视频，在大规模用户评估中表现出比 Runway、Pika 1.0、Morph、Moon Valley 和 Stable Video Diffusion 模型更优秀的性能。