General image super-resolution techniques have difficulties in recovering
detailed face structures when applying to low resolution face images. Recent
deep learning based methods tailored for face images have achieved improved
performance by jointly trained with additional task such as face parsing and
landmark prediction. However, multi-task learning requires extra manually
labeled data. Besides, most of the existing works can only generate relatively
low resolution face images (e.g., $128\times128$), and their applications are
therefore limited. In this paper, we introduce a novel SPatial Attention
Residual Network (SPARNet) built on our newly proposed Face Attention Units
(FAUs) for face super-resolution. Specifically, we introduce a spatial
attention mechanism to the vanilla residual blocks. This enables the
convolutional layers to adaptively bootstrap features related to the key face
structures and pay less attention to those less feature-rich regions. This
makes the training more effective and efficient as the key face structures only
account for a very small portion of the face image. Visualization of the
attention maps shows that our spatial attention network can capture the key
face structures well even for very low resolution faces (e.g., $16\times16$).
Quantitative comparisons on various kinds of metrics (including PSNR, SSIM,
identity similarity, and landmark detection) demonstrate the superiority of our
method over current state-of-the-arts. We further extend SPARNet with
multi-scale discriminators, named as SPARNetHD, to produce high resolution
results (i.e., $512\times512$). We show that SPARNetHD trained with synthetic
data cannot only produce high quality and high resolution outputs for
synthetically degraded face images, but also show good generalization ability
to real world low quality face images.

本篇论文介绍了一种基于 SPARNet 和 Face Attention Units（FAU）的空间注意力残差网络（SPARNet），能够在处理低分辨率人脸图像时捕捉到关键面部结构并生成高质量和高分辨率的结果。研究表明，该方法在多种度量标准上的表现优于当前最先进的方法，并能够对合成和真实世界低质量人脸图像进行有效泛化，不需要额外的人工标记数据。