The remarkable generative capabilities of diffusion models have motivated
extensive research in both image and video editing. Compared to video editing
which faces additional challenges in the time dimension, image editing has
witnessed the development of more diverse, high-quality approaches and more
capable software like Photoshop. In light of this gap, we introduce a novel and
generic solution that extends the applicability of image editing tools to
videos by propagating edits from a single frame to the entire video using a
pre-trained image-to-video model. Our method, dubbed I2VEdit, adaptively
preserves the visual and motion integrity of the source video depending on the
extent of the edits, effectively handling global edits, local edits, and
moderate shape changes, which existing methods cannot fully achieve. At the
core of our method are two main processes: Coarse Motion Extraction to align
basic motion patterns with the original video, and Appearance Refinement for
precise adjustments using fine-grained attention matching. We also incorporate
a skip-interval strategy to mitigate quality degradation from auto-regressive
generation across multiple video clips. Experimental results demonstrate our
framework's superior performance in fine-grained video editing, proving its
capability to produce high-quality, temporally consistent outputs.

通过使用预训练的图像到视频模型，我们介绍了一种新颖而通用的方法，将图像编辑工具的适用范围扩展到视频中，通过从单个帧到整个视频的传播编辑来处理全局编辑，局部编辑和适度形状变化，并且通过细粒度的注意力匹配来实现精确的调整，同时采用跳跃间隔策略来减轻多个视频片段之间自回归生成所带来的质量退化。实验结果证明了我们的框架在细粒度视频编辑方面的优异性能，并证明其能够产生高质量、时间上连续的输出。

I2VEdit: 通过图像到视频扩散模型进行首帧引导视频编辑

I2VEdit: First-Frame-Guided Video Editing via Image-to-Video Diffusion  Models

We introduce bounded generation as a generalized task to control video
generation to synthesize arbitrary camera and subject motion based only on a
given start and end frame. Our objective is to fully leverage the inherent
generalization capability of an image-to-video model without additional
training or fine-tuning of the original model. This is achieved through the
proposed new sampling strategy, which we call Time Reversal Fusion, that fuses
the temporally forward and backward denoising paths conditioned on the start
and end frame, respectively. The fused path results in a video that smoothly
connects the two frames, generating inbetweening of faithful subject motion,
novel views of static scenes, and seamless video looping when the two bounding
frames are identical. We curate a diverse evaluation dataset of image pairs and
compare against the closest existing methods. We find that Time Reversal Fusion
outperforms related work on all subtasks, exhibiting the ability to generate
complex motions and 3D-consistent views guided by bounded frames. See project
page at this https URL

我们引入有限生成作为一种广义控制视频生成的任务，仅基于给定的起始和结束帧来合成任意的摄像机和主题运动，通过新提出的 Time Reversal Fusion 采样策略完全充分利用图像到视频模型的内在泛化能力，无需额外的训练或原始模型的微调，生成平滑连接两个帧的视频，生成忠实的主题运动中间形态，静态场景的新视角，以及在两个边界帧相同时无缝视频循环。我们构建了一个多样化的评估数据集，并与最接近的现有方法进行比较，发现 Time Reversal Fusion 在所有子任务上表现优于相关工作，具有生成由有限帧引导的复杂运动和三维一致视图的能力。详见此 https URL 的项目页面。