Recent diffusion-based human image animation techniques have demonstrated
impressive success in synthesizing videos that faithfully follow a given
reference identity and a sequence of desired movement poses. Despite this,
there are still two limitations: i) an extra reference model is required to
align the identity image with the main video branch, which significantly
increases the optimization burden and model parameters; ii) the generated video
is usually short in time (e.g., 24 frames), hampering practical applications.
To address these shortcomings, we present a UniAnimate framework to enable
efficient and long-term human video generation. First, to reduce the
optimization difficulty and ensure temporal coherence, we map the reference
image along with the posture guidance and noise video into a common feature
space by incorporating a unified video diffusion model. Second, we propose a
unified noise input that supports random noised input as well as first frame
conditioned input, which enhances the ability to generate long-term video.
Finally, to further efficiently handle long sequences, we explore an
alternative temporal modeling architecture based on state space model to
replace the original computation-consuming temporal Transformer. Extensive
experimental results indicate that UniAnimate achieves superior synthesis
results over existing state-of-the-art counterparts in both quantitative and
qualitative evaluations. Notably, UniAnimate can even generate highly
consistent one-minute videos by iteratively employing the first frame
conditioning strategy. Code and models will be publicly available. Project
page: this https URL

通过引入 UniAnimate 框架，我们解决了人类图像动画技术中的两个限制，实现了高效和长期的视频生成，具备了优于现有技术的合成结果。