Deep neural networks have greatly promoted the performance of single image
super-resolution (SISR). Conventional methods still resort to restoring the
single high-resolution (HR) solution only based on the input of image modality.
However, the image-level information is insufficient to predict adequate
details and photo-realistic visual quality facing large upscaling factors (x8,
x16). In this paper, we propose a new perspective that regards the SISR as a
semantic image detail enhancement problem to generate semantically reasonable
HR image that are faithful to the ground truth. To enhance the semantic
accuracy and the visual quality of the reconstructed image, we explore the
multi-modal fusion learning in SISR by proposing a Text-Guided Super-Resolution
(TGSR) framework, which can effectively utilize the information from the text
and image modalities. Different from existing methods, the proposed TGSR could
generate HR image details that match the text descriptions through a
coarse-to-fine process. Extensive experiments and ablation studies demonstrate
the effect of the TGSR, which exploits the text reference to recover realistic
images.

本文提出了一种基于语义图像细节增强的图像超分辨率方法，其中利用文本和图像模态的多模态融合学习来提高图像的语义准确性和视觉质量。该方法通过粗到细的过程生成文本描述匹配的高分辨率图像细节，实验证明该方法具有显著的效果。