In this paper, a novel convolutional neural network (CNN)-based framework is
developed for light field reconstruction from a sparse set of views. We
indicate that the reconstruction can be efficiently modeled as angular
restoration on an epipolar plane image (EPI). The main problem in direct
reconstruction on the EPI involves an information asymmetry between the spatial
and angular dimensions, where the detailed portion in the angular dimensions is
damaged by undersampling. Directly upsampling or super-resolving the light
field in the angular dimensions causes ghosting effects. To suppress these
ghosting effects, we contribute a novel "blur-restoration-deblur" framework.
First, the "blur" step is applied to extract the low-frequency components of
the light field in the spatial dimensions by convolving each EPI slice with a
selected blur kernel. Then, the "restoration" step is implemented by a CNN,
which is trained to restore the angular details of the EPI. Finally, we use a
non-blind "deblur" operation to recover the spatial high frequencies suppressed
by the EPI blur. We evaluate our approach on several datasets, including
synthetic scenes, real-world scenes and challenging microscope light field
data. We demonstrate the high performance and robustness of the proposed
framework compared with state-of-the-art algorithms. We further show extended
applications, including depth enhancement and interpolation for unstructured
input. More importantly, a novel rendering approach is presented by combining
the proposed framework and depth information to handle large disparities.

本文提出了一种基于卷积神经网络的框架，用于从稀疏视图中重建光场；该框架将重建视为在极线平面图像上的角度恢复；通过一种 “模糊 - 恢复 - 去模糊” 的新框架，可以有效压制重叠效应，并展示应用领域，包括深度增强和插值。

基于 EPI 的卷积网络光场重构及其扩展应用

Light Field Reconstruction Using Convolutional Network on EPI and  Extended Applications

We consider the problem of high-dimensional light field reconstruction and
develop a learning-based framework for spatial and angular super-resolution.
Many current approaches either require disparity clues or restore the spatial
and angular details separately. Such methods have difficulties with
non-Lambertian surfaces or occlusions. In contrast, we formulate light field
super-resolution (LFSR) as tensor restoration and develop a learning framework
based on a two-stage restoration with 4-dimensional (4D) convolution. This
allows our model to learn the features capturing the geometry information
encoded in multiple adjacent views. Such geometric features vary near the
occlusion regions and indicate the foreground object border. To train a
feasible network, we propose a novel normalization operation based on a group
of views in the feature maps, design a stage-wise loss function, and develop
the multi-range training strategy to further improve the performance.
Evaluations are conducted on a number of light field datasets including
real-world scenes, synthetic data, and microscope light fields. The proposed
method achieves superior performance and less execution time comparing with
other state-of-the-art schemes.

利用基于张量恢复的学习框架进行高维光场重建和超分辨率，并通过多范围训练策略，提出一种新的归一化操作和阶段性损失函数来改善性能。在多个光场数据集中，该方法实现了优越的性能和更少的执行时间。

基于高维稠密残差卷积神经网络的光场重建

High-dimensional Dense Residual Convolutional Neural Network for Light  Field Reconstruction

In this article we develop an image based rendering technique based on light
field reconstruction from a limited set of perspective views acquired by
cameras. Our approach utilizes sparse representation of epipolar-plane images
in a directionally sensitive transform domain, obtained by an adapted discrete
shearlet transform. The used iterative thresholding algorithm provides
high-quality reconstruction results for relatively big disparities between
neighboring views. The generated densely sampled light field of a given 3D
scene is thus suitable for all applications which requires light field
reconstruction. The proposed algorithm is compared favorably against state of
the art depth image based rendering techniques.

本文提出了一种基于图像的渲染技术，它基于通过相机获取的有限透视视图集合的光场重建。本文的方法利用了在方向敏感变换域内的极线平面图像的稀疏表示，该变换域通过自适应离散剪切变换得到。使用的迭代阈值算法为相邻视图之间的较大差异提供高质量重建结果。因此，所生成的给定 3D 场景的密集采样光场适用于所有需要光场重建的应用程序。本文提出的算法在深度图像渲染技术方面相对于现有技术具有优势。