Despite impressive advancements in diffusion-based video editing models in
altering video attributes, there has been limited exploration into modifying
motion information while preserving the original protagonist's appearance and
background. In this paper, we propose MotionFollower, a lightweight
score-guided diffusion model for video motion editing. To introduce conditional
controls to the denoising process, MotionFollower leverages two of our proposed
lightweight signal controllers, one for poses and the other for appearances,
both of which consist of convolution blocks without involving heavy attention
calculations. Further, we design a score guidance principle based on a
two-branch architecture, including the reconstruction and editing branches,
which significantly enhance the modeling capability of texture details and
complicated backgrounds. Concretely, we enforce several consistency
regularizers and losses during the score estimation. The resulting gradients
thus inject appropriate guidance to the intermediate latents, forcing the model
to preserve the original background details and protagonists' appearances
without interfering with the motion modification. Experiments demonstrate the
competitive motion editing ability of MotionFollower qualitatively and
quantitatively. Compared with MotionEditor, the most advanced motion editing
model, MotionFollower achieves an approximately 80% reduction in GPU memory
while delivering superior motion editing performance and exclusively supporting
large camera movements and actions.

提出了一种用于视频运动编辑的轻量级评分引导扩散模型 MotionFollower，通过引入条件控制实现去噪过程，利用两个轻量级信号控制器进行姿势和外貌控制，设计了基于双分支架构的评分引导原则，维护背景和角色的外貌并改变运动细节，实验证明了 MotionFollower 在运动编辑上具有卓越的性能。

MotionFollower: 通过轻量级评分引导扩散编辑视频动作

MotionFollower: Editing Video Motion via Lightweight Score-Guided  Diffusion

Recent advances in text-guided video editing have showcased promising results
in appearance editing (e.g., stylization). However, video motion editing in the
temporal dimension (e.g., from eating to waving), which distinguishes video
editing from image editing, is underexplored. In this work, we present UniEdit,
a tuning-free framework that supports both video motion and appearance editing
by harnessing the power of a pre-trained text-to-video generator within an
inversion-then-generation framework. To realize motion editing while preserving
source video content, based on the insights that temporal and spatial
self-attention layers encode inter-frame and intra-frame dependency
respectively, we introduce auxiliary motion-reference and reconstruction
branches to produce text-guided motion and source features respectively. The
obtained features are then injected into the main editing path via temporal and
spatial self-attention layers. Extensive experiments demonstrate that UniEdit
covers video motion editing and various appearance editing scenarios, and
surpasses the state-of-the-art methods. Our code will be publicly available.

UniEdit 是一个无需调整参数的框架，它利用预先训练的文本到视频生成器，通过反演和生成的方式来支持视频运动和外观编辑。通过引入辅助运动参考和重建分支，利用时间和空间自注意层来实现运动编辑并保留源视频内容，UniEdit 在视频运动编辑和各种外观编辑场景上优于现有的方法。