Since rainy weather always degrades image quality and poses significant
challenges to most computer vision-based intelligent systems, image de-raining
has been a hot research topic. Fortunately, in a rainy light field (LF) image,
background obscured by rain streaks in one sub-view may be visible in the other
sub-views, and implicit depth information and recorded 4D structural
information may benefit rain streak detection and removal. However, existing LF
image rain removal methods either do not fully exploit the global correlations
of 4D LF data or only utilize partial sub-views, resulting in sub-optimal rain
removal performance and no-equally good quality for all de-rained sub-views. In
this paper, we propose an efficient network, called MDeRainNet, for rain streak
removal from LF images. The proposed network adopts a multi-scale
encoder-decoder architecture, which directly works on Macro-pixel images (MPIs)
to improve the rain removal performance. To fully model the global correlation
between the spatial and the angular information, we propose an Extended
Spatial-Angular Interaction (ESAI) module to merge them, in which a simple and
effective Transformer-based Spatial-Angular Interaction Attention (SAIA) block
is also proposed for modeling long-range geometric correlations and making full
use of the angular information. Furthermore, to improve the generalization
performance of our network on real-world rainy scenes, we propose a novel
semi-supervised learning framework for our MDeRainNet, which utilizes
multi-level KL loss to bridge the domain gap between features of synthetic and
real-world rain streaks and introduces colored-residue image guided contrastive
regularization to reconstruct rain-free images. Extensive experiments conducted
on synthetic and real-world LFIs demonstrate that our method outperforms the
state-of-the-art methods both quantitatively and qualitatively.

我们提出了一种名为 MDeRainNet 的高效网络，用于去除光场图像中的雨线，该网络采用多尺度编码器 - 解码器架构，在 Macro-pixel 图像上直接进行操作以提高雨线去除性能，并且借助 Extended Spatial-Angular Interaction (ESAI) 模块来完全建模空间和角度信息之间的全局相关性，并引入半监督学习框架以改善网络在现实场景中的泛化性能。经过对合成和真实光场图像的广泛实验证明，我们的方法在定量和定性上均优于现有方法。