Diffusion-based generative models have emerged as powerful tools in the realm
of generative modeling. Despite extensive research on denoising across various
timesteps and noise levels, a conflict persists regarding the relative
difficulties of the denoising tasks. While various studies argue that lower
timesteps present more challenging tasks, others contend that higher timesteps
are more difficult. To address this conflict, our study undertakes a
comprehensive examination of task difficulties, focusing on convergence
behavior and changes in relative entropy between consecutive probability
distributions across timesteps. Our observational study reveals that denoising
at earlier timesteps poses challenges characterized by slower convergence and
higher relative entropy, indicating increased task difficulty at these lower
timesteps. Building on these observations, we introduce an easy-to-hard
learning scheme, drawing from curriculum learning, to enhance the training
process of diffusion models. By organizing timesteps or noise levels into
clusters and training models with descending orders of difficulty, we
facilitate an order-aware training regime, progressing from easier to harder
denoising tasks, thereby deviating from the conventional approach of training
diffusion models simultaneously across all timesteps. Our approach leads to
improved performance and faster convergence by leveraging the benefits of
curriculum learning, while maintaining orthogonality with existing improvements
in diffusion training techniques. We validate these advantages through
comprehensive experiments in image generation tasks, including unconditional,
class-conditional, and text-to-image generation.

通过对任务困难度、收敛行为和概率分布间相对熵的变化进行综合观察，我们发现较早的时间步骤的去噪任务存在收敛缓慢和相对熵高的挑战，因此我们提出了一个易难混合的学习方案，借鉴了课程学习的思想，在训练过程中将时间步骤或噪声水平划分为难度递减的簇，并按顺序进行模型训练，以从易到难进行去噪任务，从而改变了同时跨越所有时间步骤训练扩散模型的传统方法。我们的方法通过利用课程学习的好处，提高了性能和收敛速度，并与扩散训练技术的现有改进保持正交性。我们通过在无条件生成、类条件生成和文本转图像生成等图像生成任务上的全面实验证实了这些优势。