In this paper, we consider two challenging issues in reference-based
super-resolution (RefSR), (i) how to choose a proper reference image, and (ii)
how to learn real-world RefSR in a self-supervised manner. Particularly, we
present a novel self-supervised learning approach for real-world image SR from
observations at dual camera zooms (SelfDZSR). Considering the popularity of
multiple cameras in modern smartphones, the more zoomed (telephoto) image can
be naturally leveraged as the reference to guide the SR of the lesser zoomed
(short-focus) image. Furthermore, SelfDZSR learns a deep network to obtain the
SR result of short-focus image to have the same resolution as the telephoto
image. For this purpose, we take the telephoto image instead of an additional
high-resolution image as the supervision information and select a center patch
from it as the reference to super-resolve the corresponding short-focus image
patch. To mitigate the effect of the misalignment between short-focus
low-resolution (LR) image and telephoto ground-truth (GT) image, we design an
auxiliary-LR generator and map the GT to an auxiliary-LR while keeping the
spatial position unchanged. Then the auxiliary-LR can be utilized to deform the
LR features by the proposed adaptive spatial transformer networks (AdaSTN), and
match the Ref features to GT. During testing, SelfDZSR can be directly deployed
to super-solve the whole short-focus image with the reference of telephoto
image. Experiments show that our method achieves better quantitative and
qualitative performance against state-of-the-arts. Codes are available at
this https URL

本文提出了新的自监督学习方法 (SelfDZSR)，用于从采用双倍相机变焦观察的真实世界图像中进行超分辨率重建，并以高分辨率图像为监督信息，以匹配其低分辨率版本的特征。