Recent advances in generative AI have significantly enhanced image and video
editing, particularly in the context of text prompt control. State-of-the-art
approaches predominantly rely on diffusion models to accomplish these tasks.
However, the computational demands of diffusion-based methods are substantial,
often necessitating large-scale paired datasets for training, and therefore
challenging the deployment in practical applications. This study addresses this
challenge by breaking down the text-based video editing process into two
separate stages. In the first stage, we leverage an existing text-to-image
diffusion model to simultaneously edit a few keyframes without additional
fine-tuning. In the second stage, we introduce an efficient model called
MaskINT, which is built on non-autoregressive masked generative transformers
and specializes in frame interpolation between the keyframes, benefiting from
structural guidance provided by intermediate frames. Our comprehensive set of
experiments illustrates the efficacy and efficiency of MaskINT when compared to
other diffusion-based methodologies. This research offers a practical solution
for text-based video editing and showcases the potential of non-autoregressive
masked generative transformers in this domain.

这项研究通过在文本为基础的视频编辑过程中分为两个阶段，利用现有的文本到图像扩散模型同时编辑一些关键帧，并引入了一种称为 MaskINT 的高效模型，该模型建立在非自回归的掩膜式生成变压器上，在关键帧之间进行帧插值，从中间帧提供的结构指导中受益。我们进行了一系列实验证明了 MaskINT 相对于其他基于扩散模型的方法的有效性和高效性。该研究为基于文本的视频编辑提供了实用的解决方案，并展示了非自回归的掩膜式生成变压器在该领域中的潜力。