Rain removal is important for improving the robustness of outdoor vision
based systems. Current rain removal methods show limitations either for complex
dynamic scenes shot from fast moving cameras, or under torrential rain fall
with opaque occlusions. We propose a novel derain algorithm, which applies
superpixel (SP) segmentation to decompose the scene into depth consistent
units. Alignment of scene contents are done at the SP level, which proves to be
robust towards rain occlusion and fast camera motion. Two alignment output
tensors, i.e., optimal temporal match tensor and sorted spatial-temporal match
tensor, provide informative clues for rain streak location and occluded
background contents to generate an intermediate derain output. These tensors
will be subsequently prepared as input features for a convolutional neural
network to restore high frequency details to the intermediate output for
compensation of mis-alignment blur. Extensive evaluations show that up to 5 dB
reconstruction PSNR advantage is achieved over state-of-the-art methods. Visual
inspection shows that much cleaner rain removal is achieved especially for
highly dynamic scenes with heavy and opaque rainfall from a fast moving camera.

通过将场景分解为深度一致的单位，并在 SP 级别对场景内容进行对齐处理，本文提出了一种新的去雨算法，该算法采用卷积神经网络将排序空间 - 时间匹配张量和一致的时间匹配张量准备用作输入特征，从而为中间去雨输出恢复高频细节。