We present a learning-based approach for removing unwanted obstructions, such
as window reflections, fence occlusions or raindrops, from a short sequence of
images captured by a moving camera. Our method leverages the motion differences
between the background and the obstructing elements to recover both layers.
Specifically, we alternate between estimating dense optical flow fields of the
two layers and reconstructing each layer from the flow-warped images via a deep
convolutional neural network. The learning-based layer reconstruction allows us
to accommodate potential errors in the flow estimation and brittle assumptions
such as brightness consistency. We show that training on synthetically
generated data transfers well to real images. Our results on numerous
challenging scenarios of reflection and fence removal demonstrate the
effectiveness of the proposed method.

本论文利用基于深度学习的方法，通过估计光流场并使用卷积神经网络，从短序列图像中去除窗户反射、栅栏遮挡和雨滴等干扰因素。该方法能够处理一些错误和脆弱假设，并在多个具有挑战性的场景中表现出不错的效果。

透过障碍物学习看见

Learning to See Through Obstructions

The exponentially increasing use of moving platforms for video capture
introduces the urgent need to develop the general background subtraction
algorithms with the capability to deal with the moving background. In this
paper, we propose a multilayer-based framework for online background
subtraction for videos captured by moving cameras. Unlike the previous
treatments of the problem, the proposed method is not restricted to binary
segmentation of background and foreground, but formulates it as a multi-label
segmentation problem by modeling multiple foreground objects in different
layers when they appear simultaneously in the scene. We assign an independent
processing layer to each foreground object, as well as the background, where
both motion and appearance models are estimated, and a probability map is
inferred using a Bayesian filtering framework. Finally, Multi-label Graph-cut
on Markov Random Field is employed to perform pixel-wise labeling. Extensive
evaluation results show that the proposed method outperforms state-of-the-art
methods on challenging video sequences.

针对移动拍摄视频中背景难以识别的问题，本文提出了一种基于多层结构的在线背景减除算法框架，将多前景物体建模并赋予拥有独立处理层，同时采用贝叶斯滤波框架推断概率图，并用马尔科夫随机场上的多标签图割算法进行像素级分割，取得了优于现有技术的结果。