In real-world environments, outdoor imaging systems are often affected by
disturbances such as rain degradation. Especially, in nighttime driving scenes,
insufficient and uneven lighting shrouds the scenes in darkness, resulting
degradation of both the image quality and visibility. Particularly, in the
field of autonomous driving, the visual perception ability of RGB sensors
experiences a sharp decline in such harsh scenarios. Additionally, driving
assistance systems suffer from reduced capabilities in capturing and discerning
the surrounding environment, posing a threat to driving safety. Single-view
information captured by single-modal sensors cannot comprehensively depict the
entire scene. To address these challenges, we developed an image de-raining
framework tailored for rainy nighttime driving scenes. It aims to remove rain
artifacts, enrich scene representation, and restore useful information.
Specifically, we introduce cooperative learning between visible and infrared
images captured by different sensors. By cross-view fusion of these
multi-source data, the scene within the images gains richer texture details and
enhanced contrast. We constructed an information cleaning module called
CleanNet as the first stage of our framework. Moreover, we designed an
information fusion module called FusionNet as the second stage to fuse the
clean visible images with infrared images. Using this stage-by-stage learning
strategy, we obtain de-rained fusion images with higher quality and better
visual perception. Extensive experiments demonstrate the effectiveness of our
proposed Cross-View Cooperative Learning (CVCL) in adverse driving scenarios in
low-light rainy environments. The proposed approach addresses the gap in the
utilization of existing rain removal algorithms in specific low-light
conditions.

针对夜间行驶场景中受雨水降解影响、图像质量和能见度下降的问题，我们开发了一个夜间行驶场景下的图像去雨框架，通过可见光和红外图像的跨视图融合，去除雨痕、丰富场景表征并恢复有用信息，实验证明了我们提出的跨视图合作学习方法在低光强雨水环境下具有较好效果，并弥补了现有去雨算法在特定低光条件下的利用缺口。

NiteDR: 跨视传感器协作学习的动态驾驶场景夜间图像去雨

NiteDR: Nighttime Image De-Raining with Cross-View Sensor Cooperative  Learning for Dynamic Driving Scenes

Varying weather conditions, including rainfall and snowfall, are generally
regarded as a challenge for computer vision algorithms. One proposed solution
to the challenges induced by rain and snowfall is to artificially remove the
rain from images or video using rain removal algorithms. It is the promise of
these algorithms that the rain-removed image frames will improve the
performance of subsequent segmentation and tracking algorithms. However, rain
removal algorithms are typically evaluated on their ability to remove synthetic
rain on a small subset of images. Currently, their behavior is unknown on
real-world videos when integrated with a typical computer vision pipeline. In
this paper, we review the existing rain removal algorithms and propose a new
dataset that consists of 22 traffic surveillance sequences under a broad
variety of weather conditions that all include either rain or snowfall. We
propose a new evaluation protocol that evaluates the rain removal algorithms on
their ability to improve the performance of subsequent segmentation, instance
segmentation, and feature tracking algorithms under rain and snow. If
successful, the de-rained frames of a rain removal algorithm should improve
segmentation performance and increase the number of accurately tracked
features. The results show that a recent single-frame-based rain removal
algorithm increases the segmentation performance by 19.7% on our proposed
dataset, but it eventually decreases the feature tracking performance and
showed mixed results with recent instance segmentation methods. However, the
best video-based rain removal algorithm improves the feature tracking accuracy
by 7.72%.

本文提出了一个新的数据集，用于评估基于计算机视觉技术中的人工去雨算法的效果，并提出了新的评估协议来评估这些算法在道路监控视频的不同天气条件下对分割、实例分割和特征跟踪算法的提高效果，研究结果表明，近期的单帧基于的雨去除算法可以在提高分割的准确性方面提高 19.7%，但会对特征跟踪造成负面影响，而最好的基于视频的雨去除算法可以提高 7.72% 的特征跟踪准确度。