Diffusion models have demonstrated impressive generative capabilities, but
their 'exposure bias' problem, described as the input mismatch between training
and sampling, lacks in-depth exploration. In this paper, we systematically
investigate the exposure bias problem in diffusion models by first analytically
modelling the sampling distribution, based on which we then attribute the
prediction error at each sampling step as the root cause of the exposure bias
issue. Furthermore, we discuss potential solutions to this issue and propose an
intuitive metric for it. Along with the elucidation of exposure bias, we
propose a simple, yet effective, training-free method called Epsilon Scaling to
alleviate the exposure bias. We show that Epsilon Scaling explicitly moves the
sampling trajectory closer to the vector field learned in the training phase by
scaling down the network output (Epsilon), mitigating the input mismatch
between training and sampling. Experiments on various diffusion frameworks
(ADM, DDPM/DDIM, LDM), unconditional and conditional settings, and
deterministic vs. stochastic sampling verify the effectiveness of our method.

扩散模型具有令人印象深刻的生成能力，但其所谓的 “曝光偏差” 问题，即训练和抽样之间的输入不匹配，缺乏深入探索。本文通过首先对抽样分布进行分析建模，然后将每个抽样步骤的预测误差归因为曝光偏差问题的根本原因，对扩散模型中的曝光偏差问题进行了系统研究。此外，我们讨论了此问题的潜在解决方案，并提出了一个直观的度量方法。除了阐明曝光偏差问题，我们还提出了一种简单但有效的无需训练的方法，称为 Epsilon Scaling，以缓解曝光偏差。我们通过缩小网络输出（Epsilon）使抽样轨迹明确地接近训练阶段学习到的向量场，减轻了训练和抽样之间的输入不匹配。针对各种扩散框架（ADM，DDPM/DDIM，LDM），无条件和有条件设置以及确定性与随机抽样的实验验证了我们方法的有效性。