Diffusion models have shown exceptional capabilities in generating realistic
videos. Yet, their training has been predominantly confined to offline
environments where models can repeatedly train on i.i.d. data to convergence.
This work explores the feasibility of training diffusion models from a
semantically continuous video stream, where correlated video frames
sequentially arrive one at a time. To investigate this, we introduce two novel
continual video generative modeling benchmarks, Lifelong Bouncing Balls and
Windows 95 Maze Screensaver, each containing over a million video frames
generated from navigating stationary environments. Surprisingly, our
experiments show that diffusion models can be effectively trained online using
experience replay, achieving performance comparable to models trained with
i.i.d. samples given the same number of gradient steps.

本研究介绍了通过在连续视频流中训练扩散模型的可行性，并通过引入两个新的持续视频生成建模基准数据集 —— 终身弹跳球和 Windows 95 迷宫屏幕保护程序，证明了扩散模型可以通过经验重放在线有效地训练，其性能与使用相同梯度步数训练的独立同分布样本的模型相当。