In recent years, generative artificial intelligence has achieved significant
advancements in the field of image generation, spawning a variety of
applications. However, video generation still faces considerable challenges in
various aspects, such as controllability, video length, and richness of
details, which hinder the application and popularization of this technology. In
this work, we propose a controllable video generation framework, dubbed
MimicMotion, which can generate high-quality videos of arbitrary length
mimicking specific motion guidance. Compared with previous methods, our
approach has several highlights. Firstly, we introduce confidence-aware pose
guidance that ensures high frame quality and temporal smoothness. Secondly, we
introduce regional loss amplification based on pose confidence, which
significantly reduces image distortion. Lastly, for generating long and smooth
videos, we propose a progressive latent fusion strategy. By this means, we can
produce videos of arbitrary length with acceptable resource consumption. With
extensive experiments and user studies, MimicMotion demonstrates significant
improvements over previous approaches in various aspects. Detailed results and
comparisons are available on our project page:
this https URL .

提出了一种可控的视频生成框架 MimicMotion，通过引入自信姿势引导、基于姿势置信度的区域损失放大和渐进式潜在融合策略，实现了生成任意长度高质量视频的目标，并在各个方面显示出与之前方法相比的显著改进。