Recent success in the field of single image super-resolution (SISR) is
achieved by optimizing deep convolutional neural networks (CNNs) in the image
space with the L1 or L2 loss. However, when trained with these loss functions,
models usually fail to recover sharp edges present in the high-resolution (HR)
images for the reason that the model tends to give a statistical average of
potential HR solutions. During our research, we observe that gradient maps of
images generated by the models trained with the L1 or L2 loss have
significantly lower variance than the gradient maps of the original
high-resolution images. In this work, we propose to alleviate the above issue
by introducing a structure-enhancing loss function, coined Gradient Variance
(GV) loss, and generate textures with perceptual-pleasant details.
Specifically, during the training of the model, we extract patches from the
gradient maps of the target and generated output, calculate the variance of
each patch and form variance maps for these two images. Further, we minimize
the distance between the computed variance maps to enforce the model to produce
high variance gradient maps that will lead to the generation of high-resolution
images with sharper edges. Experimental results show that the GV loss can
significantly improve both Structure Similarity (SSIM) and peak signal-to-noise
ratio (PSNR) performance of existing image super-resolution (SR) deep learning
models.

本文提出了一种名为 Gradient Variance (GV) loss 的结构增强的损失函数，用于训练模型生成高分辨率图像，以解决使用 L1 或 L2 损失函数时模型无法恢复高分辨率图像中锐利边缘的问题。实验结果表明，GV 损失可以显著提高现有图像超分辨率深度学习模型的结构相似性 (SSIM) 和峰值信噪比 (PSNR) 性能。