Transformers-based methods have achieved significant performance in image
deraining as they can model the non-local information which is vital for
high-quality image reconstruction. In this paper, we find that most existing
Transformers usually use all similarities of the tokens from the query-key
pairs for the feature aggregation. However, if the tokens from the query are
different from those of the key, the self-attention values estimated from these
tokens also involve in feature aggregation, which accordingly interferes with
the clear image restoration. To overcome this problem, we propose an effective
DeRaining network, Sparse Transformer (DRSformer) that can adaptively keep the
most useful self-attention values for feature aggregation so that the
aggregated features better facilitate high-quality image reconstruction.
Specifically, we develop a learnable top-k selection operator to adaptively
retain the most crucial attention scores from the keys for each query for
better feature aggregation. Simultaneously, as the naive feed-forward network
in Transformers does not model the multi-scale information that is important
for latent clear image restoration, we develop an effective mixed-scale
feed-forward network to generate better features for image deraining. To learn
an enriched set of hybrid features, which combines local context from CNN
operators, we equip our model with mixture of experts feature compensator to
present a cooperation refinement deraining scheme. Extensive experimental
results on the commonly used benchmarks demonstrate that the proposed method
achieves favorable performance against state-of-the-art approaches. The source
code and trained models are available at
this https URL

本论文提出了一种名为 Sparse Transformer (DRSformer) 的去雨神经网络，它可以自适应地保留最有用的自注意力值以更好地促进高质量图像重建，并发展了一种有效的混合尺度前馈网络来为图像去雨生成更好的特征。