Image-to-video (I2V) generation aims to use the initial frame (alongside a
text prompt) to create a video sequence. A grand challenge in I2V generation is
to maintain visual consistency throughout the video: existing methods often
struggle to preserve the integrity of the subject, background, and style from
the first frame, as well as ensure a fluid and logical progression within the
video narrative. To mitigate these issues, we propose ConsistI2V, a
diffusion-based method to enhance visual consistency for I2V generation.
Specifically, we introduce (1) spatiotemporal attention over the first frame to
maintain spatial and motion consistency, (2) noise initialization from the
low-frequency band of the first frame to enhance layout consistency. These two
approaches enable ConsistI2V to generate highly consistent videos. We also
extend the proposed approaches to show their potential to improve consistency
in auto-regressive long video generation and camera motion control. To verify
the effectiveness of our method, we propose I2V-Bench, a comprehensive
evaluation benchmark for I2V generation. Our automatic and human evaluation
results demonstrate the superiority of ConsistI2V over existing methods.

使用扩散方法提升图像到视频的视觉一致性，通过引入空时注意力和从低频带噪声初始化的方式，生成高度一致的视频。同时，根据提出的方法进行了自动生成长视频和相机运动控制方面的拓展，并通过 I2V-Bench 进行了评估，结果表明 ConsistI2V 方法的优越性。