Rain removal is important for improving the robustness of outdoor vision
based systems. Current rain removal methods show limitations either for complex
dynamic scenes shot from fast moving cameras, or under torrential rain fall
with opaque occlusions. We propose a novel derain algorithm, which applies
superpixel (SP) segmentation to decompose the scene into depth consistent
units. Alignment of scene contents are done at the SP level, which proves to be
robust towards rain occlusion and fast camera motion. Two alignment output
tensors, i.e., optimal temporal match tensor and sorted spatial-temporal match
tensor, provide informative clues for rain streak location and occluded
background contents to generate an intermediate derain output. These tensors
will be subsequently prepared as input features for a convolutional neural
network to restore high frequency details to the intermediate output for
compensation of mis-alignment blur. Extensive evaluations show that up to 5 dB
reconstruction PSNR advantage is achieved over state-of-the-art methods. Visual
inspection shows that much cleaner rain removal is achieved especially for
highly dynamic scenes with heavy and opaque rainfall from a fast moving camera.

通过将场景分解为深度一致的单位，并在 SP 级别对场景内容进行对齐处理，本文提出了一种新的去雨算法，该算法采用卷积神经网络将排序空间 - 时间匹配张量和一致的时间匹配张量准备用作输入特征，从而为中间去雨输出恢复高频细节。

一种基于 CNN 框架的雨水去除视频内容的鲁棒对准与补偿方法

Robust Video Content Alignment and Compensation for Rain Removal in a  CNN Framework

Single image super resolution (SISR) is to reconstruct a high resolution
image from a single low resolution image. The SISR task has been a very
attractive research topic over the last two decades. In recent years,
convolutional neural network (CNN) based models have achieved great performance
on SISR task. Despite the breakthroughs achieved by using CNN models, there are
still some problems remaining unsolved, such as how to recover high frequency
details of high resolution images. Previous CNN based models always use a pixel
wise loss, such as l2 loss. Although the high resolution images constructed by
these models have high peak signal-to-noise ratio (PSNR), they often tend to be
blurry and lack high-frequency details, especially at a large scaling factor.
In this paper, we build a super resolution perceptual generative adversarial
network (SRPGAN) framework for SISR tasks. In the framework, we propose a
robust perceptual loss based on the discriminator of the built SRPGAN model. We
use the Charbonnier loss function to build the content loss and combine it with
the proposed perceptual loss and the adversarial loss. Compared with other
state-of-the-art methods, our method has demonstrated great ability to
construct images with sharp edges and rich details. We also evaluate our method
on different benchmarks and compare it with previous CNN based methods. The
results show that our method can achieve much higher structural similarity
index (SSIM) scores on most of the benchmarks than the previous state-of-art
methods.

本文介绍了一种使用 Charbonnier Loss 函数构建内容损失，并将其与提出的感知损失和对抗性损失相结合的超分辨率感知生成对抗网络 (SRPGAN) 框架，能够更好地重建具有高频细节和清晰边缘的高分辨率图像。与其他现有模型相比，该模型在大多数基准测试中都能取得更高的结构相似性指数 (SSIM) 分数。