In this paper, a novel convolutional neural network (CNN)-based framework is
developed for light field reconstruction from a sparse set of views. We
indicate that the reconstruction can be efficiently modeled as angular
restoration on an epipolar plane image (EPI). The main problem in direct
reconstruction on the EPI involves an information asymmetry between the spatial
and angular dimensions, where the detailed portion in the angular dimensions is
damaged by undersampling. Directly upsampling or super-resolving the light
field in the angular dimensions causes ghosting effects. To suppress these
ghosting effects, we contribute a novel "blur-restoration-deblur" framework.
First, the "blur" step is applied to extract the low-frequency components of
the light field in the spatial dimensions by convolving each EPI slice with a
selected blur kernel. Then, the "restoration" step is implemented by a CNN,
which is trained to restore the angular details of the EPI. Finally, we use a
non-blind "deblur" operation to recover the spatial high frequencies suppressed
by the EPI blur. We evaluate our approach on several datasets, including
synthetic scenes, real-world scenes and challenging microscope light field
data. We demonstrate the high performance and robustness of the proposed
framework compared with state-of-the-art algorithms. We further show extended
applications, including depth enhancement and interpolation for unstructured
input. More importantly, a novel rendering approach is presented by combining
the proposed framework and depth information to handle large disparities.

本文提出了一种基于卷积神经网络的框架，用于从稀疏视图中重建光场；该框架将重建视为在极线平面图像上的角度恢复；通过一种 “模糊 - 恢复 - 去模糊” 的新框架，可以有效压制重叠效应，并展示应用领域，包括深度增强和插值。

基于 EPI 的卷积网络光场重构及其扩展应用

Light Field Reconstruction Using Convolutional Network on EPI and  Extended Applications

Light field cameras record not only the spatial information of observed
scenes but also the directions of all incoming light rays. The spatial and
angular information implicitly contain geometrical characteristics such as
multi-view or epipolar geometry, which can be exploited to improve the
performance of depth estimation. An Epipolar Plane Image (EPI), the unique 2D
spatial-angular slice of the light field, contains patterns of oriented lines.
The slope of these lines is associated with the disparity. Benefiting from this
property of EPIs, some representative methods estimate depth maps by analyzing
the disparity of each line in EPIs. However, these methods often extract the
optimal slope of the lines from EPIs while ignoring the relationship between
neighboring pixels, which leads to inaccurate depth map predictions. Based on
the observation that an oriented line and its neighboring pixels in an EPI
share a similar linear structure, we propose an end-to-end fully convolutional
network (FCN) to estimate the depth value of the intersection point on the
horizontal and vertical EPIs. Specifically, we present a new feature-extraction
module, called Oriented Relation Module (ORM), that constructs the relationship
between the line orientations. To facilitate training, we also propose a
refocusing-based data augmentation method to obtain different slopes from EPIs
of the same scene point. Extensive experiments verify the efficacy of learning
relations and show that our approach is competitive to other state-of-the-art
methods. The code and the trained models are available at
this https URL

本文提出了一种基于全卷积网络的深度估计方法，用于通过利用光场相机获取的空间和角度信息中的几何特征，估计场景的深度图，并引入了 “定向关系模块” 帮助建立直线方向之间的关系来提高估计的精度。