We present a novel task called online video editing, which is designed to
edit \textbf{streaming} frames while maintaining temporal consistency. Unlike
existing offline video editing assuming all frames are pre-established and
accessible, online video editing is tailored to real-life applications such as
live streaming and online chat, requiring (1) fast continual step inference,
(2) long-term temporal modeling, and (3) zero-shot video editing capability. To
solve these issues, we propose Streaming Video Diffusion (SVDiff), which
incorporates the compact spatial-aware temporal recurrence into off-the-shelf
Stable Diffusion and is trained with the segment-level scheme on large-scale
long videos. This simple yet effective setup allows us to obtain a single model
that is capable of executing a broad range of videos and editing each streaming
frame with temporal coherence. Our experiments indicate that our model can edit
long, high-quality videos with remarkable results, achieving a real-time
inference speed of 15.2 FPS at a resolution of 512x512.

我们提出了一种名为在线视频编辑的新任务，旨在在保持时间一致性的同时编辑流式帧。我们提出了 Streaming Video Diffusion（SVDiff）来解决此问题，它将紧凑的空间感知时间回归与现成的稳定扩散相结合，并采用分段级方案在大规模长视频上进行训练。通过这种简单而有效的设置，我们可以获得一个能够执行各种视频并具有时序连续性的单一模型。我们的实验表明，我们的模型能够以 512x512 的分辨率实现 15.2 FPS 的实时推理速度，对于长时间、高质量的视频编辑表现出色。