Diffusion models have achieved remarkable success in generating high-quality
images thanks to their novel training procedures applied to unprecedented
amounts of data. However, training a diffusion model from scratch is
computationally expensive. This highlights the need to investigate the
possibility of training these models iteratively, reusing computation while the
data distribution changes. In this study, we take the first step in this
direction and evaluate the continual learning (CL) properties of diffusion
models. We begin by benchmarking the most common CL methods applied to
Denoising Diffusion Probabilistic Models (DDPMs), where we note the strong
performance of the experience replay with the reduced rehearsal coefficient.
Furthermore, we provide insights into the dynamics of forgetting, which exhibit
diverse behavior across diffusion timesteps. We also uncover certain pitfalls
of using the bits-per-dimension metric for evaluating CL.

本研究评估扩散模型的连续学习特性，通过给 Denoising Diffusion Probabilistic Models （DDPMs）应用经验重放，发现一定程度的遗忘可以被减少，但在扩散步数上，其表现多种多样。同时揭示使用 bits-per-dimension 指标评估 CL 时的某些问题。