Diffusion-based super-resolution (SR) models have recently garnered
significant attention due to their potent restoration capabilities. But
conventional diffusion models perform noise sampling from a single
distribution, constraining their ability to handle real-world scenes and
complex textures across semantic regions. With the success of segment anything
model (SAM), generating sufficiently fine-grained region masks can enhance the
detail recovery of diffusion-based SR model. However, directly integrating SAM
into SR models will result in much higher computational cost. In this paper, we
propose the SAM-DiffSR model, which can utilize the fine-grained structure
information from SAM in the process of sampling noise to improve the image
quality without additional computational cost during inference. In the process
of training, we encode structural position information into the segmentation
mask from SAM. Then the encoded mask is integrated into the forward diffusion
process by modulating it to the sampled noise. This adjustment allows us to
independently adapt the noise mean within each corresponding segmentation area.
The diffusion model is trained to estimate this modulated noise. Crucially, our
proposed framework does NOT change the reverse diffusion process and does NOT
require SAM at inference. Experimental results demonstrate the effectiveness of
our proposed method, showcasing superior performance in suppressing artifacts,
and surpassing existing diffusion-based methods by 0.74 dB at the maximum in
terms of PSNR on DIV2K dataset. The code and dataset are available at
this https URL

我们提出了 SAM-DiffSR 模型，通过将细粒度结构信息编码到分割掩膜中，并在正向扩散过程中调制噪声，实现了对图像质量的改善，同时不增加推理过程中的计算成本。在 DIV2K 数据集上的实验结果表明，我们的方法在抑制伪影方面表现出卓越的性能，并且在 PSNR 方面超过现有的基于扩散的方法高达 0.74 dB。

SAM-DiffSR：图像超分辨率的结构调制扩散模型

SAM-DiffSR: Structure-Modulated Diffusion Model for Image  Super-Resolution

Limited angle problem is a challenging issue in x-ray computed tomography
(CT) field. Iterative reconstruction methods that utilize the additional prior
can suppress artifacts and improve image quality, but unfortunately require
increased computation time. An interesting way is to restrain the artifacts in
the images reconstructed from the practical filtered back projection (FBP)
method. Frikel and Quinto have proved that the streak artifacts in FBP results
could be characterized. It indicates that the artifacts created by FBP method
have specific and similar characteristics in a stationary limited-angle
scanning configuration. Based on this understanding, this work aims at
developing a method to extract and suppress specific artifacts of FBP
reconstructions for limited-angle tomography. A data-driven learning-based
method is proposed based on a deep convolutional neural network. An end-to-end
mapping between the FBP and artifact-free images is learned and the implicit
features involving artifacts will be extracted and suppressed via nonlinear
mapping. The qualitative and quantitative evaluations of experimental results
indicate that the proposed method show a stable and prospective performance on
artifacts reduction and detail recovery for limited angle tomography. The
presented strategy provides a simple and efficient approach for improving image
quality of the reconstruction results from limited projection data.

本文提出了一种基于深度卷积神经网络的数据驱动学习方法，用于在限角摄影术下提取和抑制 FBP 重建的特定伪影。 实验结果的定性和定量评估表明，该方法在伪影抑制和细节恢复方面表现出稳定和有前途的性能，为从有限投影数据中重构出的图像质量的提高提供了简单和高效的途径。