In this paper, we introduce YONOS-SR, a novel stable diffusion-based approach
for image super-resolution that yields state-of-the-art results using only a
single DDIM step. We propose a novel scale distillation approach to train our
SR model. Instead of directly training our SR model on the scale factor of
interest, we start by training a teacher model on a smaller magnification
scale, thereby making the SR problem simpler for the teacher. We then train a
student model for a higher magnification scale, using the predictions of the
teacher as a target during the training. This process is repeated iteratively
until we reach the target scale factor of the final model. The rationale behind
our scale distillation is that the teacher aids the student diffusion model
training by i) providing a target adapted to the current noise level rather
than using the same target coming from ground truth data for all noise levels
and ii) providing an accurate target as the teacher has a simpler task to
solve. We empirically show that the distilled model significantly outperforms
the model trained for high scales directly, specifically with few steps during
inference. Having a strong diffusion model that requires only one step allows
us to freeze the U-Net and fine-tune the decoder on top of it. We show that the
combination of spatially distilled U-Net and fine-tuned decoder outperforms
state-of-the-art methods requiring 200 steps with only one single step.

本文介绍了 YONOS-SR，一种新的稳定的基于扩散的图像超分辨率方法，仅使用一次 DDIM 步骤即可获得最先进的结果。通过提出一种新颖的尺度蒸馏方法来训练我们的 SR 模型，我们将 SR 问题简化为教师模型在较小放大尺度上的训练。在训练过程中，我们使用教师的预测作为目标来训练较高放大尺度的学生模型，并通过迭代重复该过程，直到达到最终模型的目标尺度。我们实验证明，蒸馏模型明显优于直接训练高放大尺度的模型，在推断过程中只需要少量步骤。拥有只需一步的强大扩散模型使我们能够冻结 U-Net，并在其上微调解码器。我们展示了经过空间蒸馏的 U-Net 和经过微调的解码器的组合比需要 200 步的最先进方法仅使用单次步骤更为优越。