Non-local attention module has been proven to be crucial for image
restoration. Conventional non-local attention processes features of each layer
separately, so it risks missing correlation between features among different
layers. To address this problem, we aim to design attention modules that
aggregate information from different layers. Instead of finding correlated key
pixels within the same layer, each query pixel is encouraged to attend to key
pixels at multiple previous layers of the network. In order to efficiently
embed such attention design into neural network backbones, we propose a novel
Adaptive Cross-Layer Attention (ACLA) module. Two adaptive designs are proposed
for ACLA: (1) adaptively selecting the keys for non-local attention at each
layer; (2) automatically searching for the insertion locations for ACLA
modules. By these two adaptive designs, ACLA dynamically selects a flexible
number of keys to be aggregated for non-local attention at previous layer while
maintaining a compact neural network with compelling performance. Extensive
experiments on image restoration tasks, including single image
super-resolution, image denoising, image demosaicing, and image compression
artifacts reduction, validate the effectiveness and efficiency of ACLA. The
code of ACLA is available at https://github.com/SDL-ASU/ACLA.

该论文提出了一种新型的自适应跨层关注模块（ACLA），来提高图像修复的效率。ACLA 为每个查询像素选择多个关键像素，并提供适配性的选择和插入设计，使其可以更好地融合不同层之间特征的相关性，从而提高图像修复的性能。