Image generation and editing have seen a great deal of advancements with the
rise of large-scale diffusion models that allow user control of different
modalities such as text, mask, depth maps, etc. However, controlled editing of
videos still lags behind. Prior work in this area has focused on using 2D
diffusion models to globally change the style of an existing video. On the
other hand, in many practical applications, editing localized parts of the
video is critical. In this work, we propose a method to edit videos using a
pre-trained inpainting image diffusion model. We systematically redesign the
forward path of the model by replacing the self-attention modules with an
extended version of attention modules that creates frame-level dependencies. In
this way, we ensure that the edited information will be consistent across all
the video frames no matter what the shape and position of the masked area is.
We qualitatively compare our results with state-of-the-art in terms of accuracy
on several video editing tasks like object retargeting, object replacement, and
object removal tasks. Simulations demonstrate the superior performance of the
proposed strategy.

使用预训练的图像修复扩散模型，通过替换自注意力模块以创建帧级依赖关系，我们提出了一种编辑视频的方法，以确保编辑信息在所有视频帧中保持一致，从而在对象重定向、对象替换和对象移除等多个视频编辑任务中展示了该策略的卓越性能。