With recent advances in image and video diffusion models for content
creation, a plethora of techniques have been proposed for customizing their
generated content. In particular, manipulating the cross-attention layers of
Text-to-Image (T2I) diffusion models has shown great promise in controlling the
shape and location of objects in the scene. Transferring image-editing
techniques to the video domain, however, is extremely challenging as object
motion and temporal consistency are difficult to capture accurately. In this
work, we take a first look at the role of cross-attention in Text-to-Video
(T2V) diffusion models for zero-shot video editing. While one-shot models have
shown potential in controlling motion and camera movement, we demonstrate
zero-shot control over object shape, position and movement in T2V models. We
show that despite the limitations of current T2V models, cross-attention
guidance can be a promising approach for editing videos.

最新的图像和视频扩散模型在内容创作方面取得了显著进展，为定制生成内容提供了大量技术。本文首次研究了跨注意力在零样本视频编辑中的作用，展示了在 T2V 模型中实现对物体形状、位置和运动的控制的潜力。

研究跨注意力对解锁零 - shot 编辑的文本到视频扩散模型的效果

Investigating the Effectiveness of Cross-Attention to Unlock Zero-Shot  Editing of Text-to-Video Diffusion Models

This paper presents a video inversion approach for zero-shot video editing,
which aims to model the input video with low-rank representation during the
inversion process. The existing video editing methods usually apply the typical
2D DDIM inversion or na\"ive spatial-temporal DDIM inversion before editing,
which leverages time-varying representation for each frame to derive noisy
latent. Unlike most existing approaches, we propose a Spatial-Temporal
Expectation-Maximization (STEM) inversion, which formulates the dense video
feature under an expectation-maximization manner and iteratively estimates a
more compact basis set to represent the whole video. Each frame applies the
fixed and global representation for inversion, which is more friendly for
temporal consistency during reconstruction and editing. Extensive qualitative
and quantitative experiments demonstrate that our STEM inversion can achieve
consistent improvement on two state-of-the-art video editing methods.

该论文提出了一种用于零样本视频编辑的视频反演方法，通过在反演过程中对输入视频建模为低秩表示。研究发现，与现有的方法不同，提出的空时期望最大化（STEM）反演方法能够在整个视频中应用固定和全局表示以保证重建和编辑的时间一致性，并取得了两种最先进的视频编辑方法上的一致改进。