This paper explores the problem of reconstructing high-resolution light field
(LF) images from hybrid lenses, including a high-resolution camera surrounded
by multiple low-resolution cameras. To tackle this challenge, we propose a
novel end-to-end learning-based approach, which can comprehensively utilize the
specific characteristics of the input from two complementary and parallel
perspectives. Specifically, one module regresses a spatially consistent
intermediate estimation by learning a deep multidimensional and cross-domain
feature representation; the other one constructs another intermediate
estimation, which maintains the high-frequency textures, by propagating the
information of the high-resolution view. We finally leverage the advantages of
the two intermediate estimations via the learned attention maps, leading to the
final high-resolution LF image. Extensive experiments demonstrate the
significant superiority of our approach over state-of-the-art ones. That is,
our method not only improves the PSNR by more than 2 dB, but also preserves the
LF structure much better. To the best of our knowledge, this is the first
end-to-end deep learning method for reconstructing a high-resolution LF image
with a hybrid input. We believe our framework could potentially decrease the
cost of high-resolution LF data acquisition and also be beneficial to LF data
storage and transmission. The code is available at
this https URL

本文利用新颖的端到端学习法解决通过混合镜头（高分辨率相机周围有多个低分辨率相机）重建高分辨率光场图像的问题，通过深度多维和跨域特征表征回归一个具有空间一致性的中间估计，同时构建另一个维护高频纹理的中间估计，在通过学习的注意图整合两个中间估计的优势，最终重建高分辨率的光场图像。实验证明本研究方法相较于现有技术在提高峰值信噪比两个单位的同时保留了光场结构，并且是首个使用端到端深度学习方法解决混合输入高分辨率光场图像问题。