Pre-trained diffusion models utilized for image generation encapsulate a substantial reservoir of a priori knowledge pertaining to intricate textures. Harnessing the potential of leveraging this a priori knowledge in the context of image super-resolution presents a compelling avenue. Nonetheless, prevailing diffusion-based methodologies presently overlook the constraints imposed by degradation information on the diffusion process. Furthermore, these methods fail to consider the spatial variability inherent in the estimated blur kernel, stemming from factors such as motion jitter and out-of-focus elements in open-environment scenarios. This oversight results in a notable deviation of the image super-resolution effect from fundamental realities. To address these concerns, we introduce a framework known as Adaptive Multi-modal Fusion of \textbf{S}patially Variant Kernel Refinement with Diffusion Model for Blind Image \textbf{S}uper-\textbf{R}esolution (SSR). Within the SSR framework, we propose a Spatially Variant Kernel Refinement (SVKR) module. SVKR estimates a Depth-Informed Kernel, which takes the depth information into account and is spatially variant. Additionally, SVKR enhance the accuracy of depth information acquired from LR images, allowing for mutual enhancement between the depth map and blur kernel estimates. Finally, we introduce the Adaptive Multi-Modal Fusion (AMF) module to align the information from three modalities: low-resolution images, depth maps, and blur kernels. This alignment can constrain the diffusion model to generate more authentic SR results. Quantitative and qualitative experiments affirm the superiority of our approach, while ablation experiments corroborate the effectiveness of the modules we have proposed.

利用预训练的扩散模型进行图像生成，包含大量关于复杂纹理的先验知识。为了解决现有方法在扩散过程中忽视退化信息约束以及空间变异性的问题，本文提出了自适应多模态融合空间变异核细化与扩散模型的盲超分辨率图像方法(SSR)。通过引入空间变异核细化模块(SVKR)，结合深度信息和模糊核估计，以及自适应多模态融合模块(AMF)对低分辨率图像、深度图和模糊核信息进行优化调整，实现更真实的超分辨率结果。实验证明了本文方法的有效性和优越性。

自适应多模态融合的空间变体核细化与扩散模型盲图像超分辨率