In this paper, we consider two challenging issues in reference-based
super-resolution (RefSR) for smartphone, (i) how to choose a proper reference
image, and (ii) how to learn RefSR in a self-supervised manner. Particularly,
we propose a novel self-supervised learning approach for real-world RefSR from
observations at dual and multiple camera zooms. Firstly, considering the
popularity of multiple cameras in modern smartphones, the more zoomed
(telephoto) image can be naturally leveraged as the reference to guide the
super-resolution (SR) of the lesser zoomed (ultra-wide) image, which gives us a
chance to learn a deep network that performs SR from the dual zoomed
observations (DZSR). Secondly, for self-supervised learning of DZSR, we take
the telephoto image instead of an additional high-resolution image as the
supervision information, and select a center patch from it as the reference to
super-resolve the corresponding ultra-wide image patch. To mitigate the effect
of the misalignment between ultra-wide low-resolution (LR) patch and telephoto
ground-truth (GT) image during training, we first adopt patch-based optical
flow alignment and then design an auxiliary-LR to guide the deforming of the
warped LR features. To generate visually pleasing results, we present local
overlapped sliced Wasserstein loss to better represent the perceptual
difference between GT and output in the feature space. During testing, DZSR can
be directly deployed to super-solve the whole ultra-wide image with the
reference of the telephoto image. In addition, we further take multiple zoomed
observations to explore self-supervised RefSR, and present a progressive fusion
scheme for the effective utilization of reference images. Experiments show that
our methods achieve better quantitative and qualitative performance against
state-of-the-arts. Codes are available at
this https URL

在这篇论文中，我们针对智能手机的参考图像超分辨率（RefSR）处理中的两个挑战性问题进行了研究，即如何选择适当的参考图像和如何以自监督的方式学习 RefSR。同时，我们提出了一种针对真实场景下的 RefSR 的自监督学习方法，通过观察双重和多重摄像头放大的图像。我们的方法在实验中表现出更好的定量和定性性能，并提供了可用的代码。