Optical flow estimation in the rainy scenes is challenging due to background
degradation introduced by rain streaks and rain accumulation effects in the
scene. Rain accumulation effect refers to poor visibility of remote objects due
to the intense rainfall. Most existing optical flow methods are erroneous when
applied to rain sequences because the conventional brightness constancy
constraint (BCC) and gradient constancy constraint (GCC) generally break down
in this situation. Based on the observation that the RGB color channels receive
raindrop radiance equally, we introduce a residue channel as a new data
constraint to reduce the effect of rain streaks. To handle rain accumulation,
our method decomposes the image into a piecewise-smooth background layer and a
high-frequency detail layer. It also enforces the BCC on the background layer
only. Results on both synthetic dataset and real images show that our algorithm
outperforms existing methods on different types of rain sequences. To our
knowledge, this is the first optical flow method specifically dealing with
rain.

通过在雨中观察到 RGB 色彩通道均接收雨点辐射，文章引入了一种新的数据约束 —— 残差通道，用于减少雨条的影响，同时将图像分解为平滑背景层和高频细节层，并仅对背景层执行亮度恒定约束，证明在各种类型的雨情况下，我们的算法优于现有方法。这是第一个专门处理雨的光流方法。