There is a bias in the inference pipeline of most diffusion models. This bias
arises from a signal leak whose distribution deviates from the noise
distribution, creating a discrepancy between training and inference processes.
We demonstrate that this signal-leak bias is particularly significant when
models are tuned to a specific style, causing sub-optimal style matching.
Recent research tries to avoid the signal leakage during training. We instead
show how we can exploit this signal-leak bias in existing diffusion models to
allow more control over the generated images. This enables us to generate
images with more varied brightness, and images that better match a desired
style or color. By modeling the distribution of the signal leak in the spatial
frequency and pixel domains, and including a signal leak in the initial latent,
we generate images that better match expected results without any additional
training.

扩散模型中存在一种偏倚，来自信号泄漏的偏差概率分布与噪声概率分布不一致，造成训练和推理过程不匹配。我们展示了当模型针对特定风格进行调整时，信号泄漏偏倚尤为显著，导致次优的风格匹配。最近的研究尝试在训练过程中避免信号泄漏，而我们则展示了如何利用现有扩散模型中的信号泄漏偏倚，以实现对生成图像的更多控制。这使我们能够生成亮度更加多变的图像，并且更好地匹配所期望的风格或颜色。通过对信号泄漏在空间频率和像素域的分布进行建模，并在初始潜在空间中引入一个信号泄漏，我们生成的图像更好地匹配预期结果，无需任何额外的训练。