In this paper, we propose an approach to address the problem of 3D
reconstruction of scenes from a single image captured by a light-field camera
equipped with a rolling shutter sensor. Our method leverages the 3D information
cues present in the light-field and the motion information provided by the
rolling shutter effect. We present a generic model for the imaging process of
this sensor and a two-stage algorithm that minimizes the re-projection error
while considering the position and motion of the camera in a motion-shape
bundle adjustment estimation strategy. Thereby, we provide an instantaneous 3D
shape-and-pose-and-velocity sensing paradigm. To the best of our knowledge,
this is the first study to leverage this type of sensor for this purpose. We
also present a new benchmark dataset composed of different light-fields showing
rolling shutter effects, which can be used as a common base to improve the
evaluation and tracking the progress in the field. We demonstrate the
effectiveness and advantages of our approach through several experiments
conducted for different scenes and types of motions. The source code and
dataset are publicly available at: this https URL

我们提出了一种方法来解决由具有滚动快门传感器的光场相机拍摄的单幅图像的三维场景重建问题。我们的方法利用光场中存在的三维信息线索和滚动快门效应提供的运动信息。我们提出了一个适用于该传感器的成像过程的通用模型和一个两阶段算法，该算法能在考虑相机的位置和运动的情况下，最小化重新投影误差，并采用运动 - 形状捆绑调整估计策略。从而，我们提供了一种瞬时的三维形状 - 姿态 - 速度感知范例。据我们所知，这是第一项利用这种传感器进行这种目的的研究。我们还提供了一个由展示滚动快门效应的不同光场组成的新的基准数据集，可用作改进该领域评估和跟踪进展的共同基础。我们通过几个针对不同场景和运动类型的实验来展示我们方法的有效性和优势。源代码和数据集可在此 https URL 公开访问。