Conventional approaches to image de-fencing have limited themselves to using
only image data in adjacent frames of the captured video of an approximately
static scene. In this work, we present a method to harness disparity using a
stereo pair of fenced images in order to detect fence pixels. Tourists and
amateur photographers commonly carry smartphones/phablets which can be used to
capture a short video sequence of the fenced scene. We model the formation of
the occluded frames in the captured video. Furthermore, we propose an
optimization framework to estimate the de-fenced image using the total
variation prior to regularize the ill-posed problem.

本文提出了一种使用视差及立体图像检测围栏像素、模拟拍摄视频中被挡住的图像帧，以及采用总变差先验正则化的优化方法来进行解围栏图像的方法。

利用智能手机实现立体图像除栏功能

Stereo image de-fencing using smartphones

Conventional approaches to image de-fencing use multiple adjacent frames for
segmentation of fences in the reference image and are limited to restoring
images of static scenes only. In this paper, we propose a de-fencing algorithm
for images of dynamic scenes using an occlusion-aware optical flow method. We
divide the problem of image de-fencing into the tasks of automated fence
segmentation from a single image, motion estimation under known occlusions and
fusion of data from multiple frames of a captured video of the scene.
Specifically, we use a pre-trained convolutional neural network to segment
fence pixels from a single image. The knowledge of spatial locations of fences
is used to subsequently estimate optical flow in the occluded frames of the
video for the final data fusion step. We cast the fence removal problem in an
optimization framework by modeling the formation of the degraded observations.
The inverse problem is solved using fast iterative shrinkage thresholding
algorithm (FISTA). Experimental results show the effectiveness of proposed
algorithm.

提出了一种基于遮挡感知光流方法的图像去护栏算法，通过将图像去护栏问题划分为从单个图像自动分割栅栏，估计已知遮挡下的运动和来自捕获视频多帧的数据融合等任务，使用预训练的卷积神经网络在单个图像中分割栅栏像素，并为最终数据融合步骤在视频的遮挡帧中估计光流。通过模拟退火算法将去护栏问题建模为优化框架，反问题使用快速迭代收缩阈值算法（FISTA）解决，实验结果表明了该算法的有效性。