We study on image super-resolution (SR), which aims to recover realistic
textures from a low-resolution (LR) image. Recent progress has been made by
taking high-resolution images as references (Ref), so that relevant textures
can be transferred to LR images. However, existing SR approaches neglect to use
attention mechanisms to transfer high-resolution (HR) textures from Ref images,
which limits these approaches in challenging cases. In this paper, we propose a
novel Texture Transformer Network for Image Super-Resolution (TTSR), in which
the LR and Ref images are formulated as queries and keys in a transformer,
respectively. TTSR consists of four closely-related modules optimized for image
generation tasks, including a learnable texture extractor by DNN, a relevance
embedding module, a hard-attention module for texture transfer, and a
soft-attention module for texture synthesis. Such a design encourages joint
feature learning across LR and Ref images, in which deep feature
correspondences can be discovered by attention, and thus accurate texture
features can be transferred. The proposed texture transformer can be further
stacked in a cross-scale way, which enables texture recovery from different
levels (e.g., from 1x to 4x magnification). Extensive experiments show that
TTSR achieves significant improvements over state-of-the-art approaches on both
quantitative and qualitative evaluations.

本文提出了一种基于注意机制的纹理转换神经网络（TTSR），采用 DNN 可学习的纹理提取器、相关嵌入模块、用于纹理传输的硬注意力模块，并且结合 LR 和 Ref 图像进行联合特征学习，从而实现图像的纹理恢复。经过广泛实验，TTSR 在定量和定性评估上都取得了显著改进。