Recent advances in diffusion models attempt to handle conditional generative
tasks by utilizing a differentiable loss function for guidance without the need
for additional training. While these methods achieved certain success, they
often compromise on sample quality and require small guidance step sizes,
leading to longer sampling processes. This paper reveals that the fundamental
issue lies in the manifold deviation during the sampling process when loss
guidance is employed. We theoretically show the existence of manifold deviation
by establishing a certain lower bound for the estimation error of the loss
guidance. To mitigate this problem, we propose Diffusion with Spherical
Gaussian constraint (DSG), drawing inspiration from the concentration
phenomenon in high-dimensional Gaussian distributions. DSG effectively
constrains the guidance step within the intermediate data manifold through
optimization and enables the use of larger guidance steps. Furthermore, we
present a closed-form solution for DSG denoising with the Spherical Gaussian
constraint. Notably, DSG can seamlessly integrate as a plugin module within
existing training-free conditional diffusion methods. Implementing DSG merely
involves a few lines of additional code with almost no extra computational
overhead, yet it leads to significant performance improvements. Comprehensive
experimental results in various conditional generation tasks validate the
superiority and adaptability of DSG in terms of both sample quality and time
efficiency.

通过使用不可微的损失函数进行指导，无需额外训练的扩散模型在处理条件生成任务方面取得一定的成功，但常常在样本质量方面有所妥协，需要较小的指导步长，从而导致较长的采样过程。本文揭示了损失指导带来的采样过程中流形偏离的根本问题，通过建立损失指导估计误差的下界理论上证明了流形偏离的存在。为了缓解这个问题，我们基于高维高斯分布中的浓度现象提出了具有球面高斯约束的扩散模型 (DSG)，通过优化将指导步骤有效限制在中间数据流形内，并实现了 DSG 去噪的闭合形式解。值得注意的是，DSG 可以作为插件模块无缝地集成到现有的无需训练的条件扩散方法中。仅需要添加几行附加代码，并几乎不带来额外的计算开销，但可以显著提高性能。基于各种条件生成任务的全面实验结果验证了 DSG 在样本质量和时间效率方面的优越性和适应性。