Non-local attention module has been proven to be crucial for image
restoration. Conventional non-local attention processes features of each layer
separately, so it risks missing correlation between features among different
layers. To address this problem, we aim to design attention modules that
aggregate information from different layers. Instead of finding correlated key
pixels within the same layer, each query pixel is encouraged to attend to key
pixels at multiple previous layers of the network. In order to efficiently
embed such attention design into neural network backbones, we propose a novel
Adaptive Cross-Layer Attention (ACLA) module. Two adaptive designs are proposed
for ACLA: (1) adaptively selecting the keys for non-local attention at each
layer; (2) automatically searching for the insertion locations for ACLA
modules. By these two adaptive designs, ACLA dynamically selects a flexible
number of keys to be aggregated for non-local attention at previous layer while
maintaining a compact neural network with compelling performance. Extensive
experiments on image restoration tasks, including single image
super-resolution, image denoising, image demosaicing, and image compression
artifacts reduction, validate the effectiveness and efficiency of ACLA. The
code of ACLA is available at https://github.com/SDL-ASU/ACLA.

该论文提出了一种新型的自适应跨层关注模块（ACLA），来提高图像修复的效率。ACLA 为每个查询像素选择多个关键像素，并提供适配性的选择和插入设计，使其可以更好地融合不同层之间特征的相关性，从而提高图像修复的性能。

自适应跨层级注意力图像修复

Adaptive Cross-Layer Attention for Image Restoration

Deep convolution-based single image super-resolution (SISR) networks embrace
the benefits of learning from large-scale external image resources for local
recovery, yet most existing works have ignored the long-range feature-wise
similarities in natural images. Some recent works have successfully leveraged
this intrinsic feature correlation by exploring non-local attention modules.
However, none of the current deep models have studied another inherent property
of images: cross-scale feature correlation. In this paper, we propose the first
Cross-Scale Non-Local (CS-NL) attention module with integration into a
recurrent neural network. By combining the new CS-NL prior with local and
in-scale non-local priors in a powerful recurrent fusion cell, we can find more
cross-scale feature correlations within a single low-resolution (LR) image. The
performance of SISR is significantly improved by exhaustively integrating all
possible priors. Extensive experiments demonstrate the effectiveness of the
proposed CS-NL module by setting new state-of-the-arts on multiple SISR
benchmarks.

本论文提出了首个可将跨尺度特征相关性与非局部注意力机制相融合的循环神经网络，该模型及其与本地和同尺度非局部注意力结合可以更好地发现单个低分辨率图像中的跨尺度特征相关性，从而显著提高图像超分辨率的性能，并在多项 SISR 基准测试中表现出新的最佳效果。