Light field (LF) depth estimation is a crucial task with numerous practical
applications. However, mainstream methods based on the multi-view stereo (MVS)
are resource-intensive and time-consuming as they need to construct a finer
cost volume. To address this issue and achieve a better trade-off between
accuracy and efficiency, we propose an occlusion-aware cascade cost volume for
LF depth (disparity) estimation. Our cascaded strategy reduces the sampling
number while keeping the sampling interval constant during the construction of
a finer cost volume. We also introduce occlusion maps to enhance accuracy in
constructing the occlusion-aware cost volume. Specifically, we first obtain the
coarse disparity map through the coarse disparity estimation network. Then, the
sub-aperture images (SAIs) of side views are warped to the center view based on
the initial disparity map. Next, we propose photo-consistency constraints
between the warped SAIs and the center SAI to generate occlusion maps for each
SAI. Finally, we introduce the coarse disparity map and occlusion maps to
construct an occlusion-aware refined cost volume, enabling the refined
disparity estimation network to yield a more precise disparity map. Extensive
experiments demonstrate the effectiveness of our method. Compared with
state-of-the-art methods, our method achieves a superior balance between
accuracy and efficiency and ranks first in terms of MSE and Q25 metrics among
published methods on the HCI 4D benchmark. The code and model of the proposed
method are available at this https URL

本研究提出了一个基于层级累加代价体积和遮挡感知技术的光场深度估计方法，该方法利用相机阵列的子孔径图像对中心图像进行提取深度，同时通过引入遮挡地图来提高深度的估计准确性。实验证明了该方法比同类方法更加精确和高效。

OccCasNet：用于光场深度估计的遮挡感知级联代价体积

OccCasNet: Occlusion-aware Cascade Cost Volume for Light Field Depth  Estimation

The success of existing deep-learning based multi-view stereo (MVS)
approaches greatly depends on the availability of large-scale supervision in
the form of dense depth maps. Such supervision, while not always possible,
tends to hinder the generalization ability of the learned models in
never-seen-before scenarios. In this paper, we propose the first unsupervised
learning based MVS network, which learns the multi-view depth maps from the
input multi-view images and does not need ground-truth 3D training data. Our
network is symmetric in predicting depth maps for all views simultaneously,
where we enforce cross-view consistency of multi-view depth maps during both
training and testing stages. Thus, the learned multi-view depth maps naturally
comply with the underlying 3D scene geometry. Besides, our network also learns
the multi-view occlusion maps, which further improves the robustness of our
network in handling real-world occlusions. Experimental results on multiple
benchmarking datasets demonstrate the effectiveness of our network and the
excellent generalization ability.

本文介绍了一种基于无监督学习的多视图深度图像学习方法，可以从多个视角的输入图像中学习深度和遮挡图像，并在训练和测试阶段都强制实施多视图深度一致性，进而为实际场景下处理遮挡提供更好的鲁棒性。

MVS^2: 多视图对称深度无监督多视图立体成像

MVS^2: Deep Unsupervised Multi-view Stereo with Multi-View Symmetry

A dynamic scene has two types of elements: those that move fluidly and can be
predicted from previous frames, and those which are disoccluded (exposed) and
cannot be extrapolated. Prior approaches to video prediction typically learn
either to warp or to hallucinate future pixels, but not both. In this paper, we
describe a computational model for high-fidelity video prediction which
disentangles motion-specific propagation from motion-agnostic generation. We
introduce a confidence-aware warping operator which gates the output of pixel
predictions from a flow predictor for non-occluded regions and from a context
encoder for occluded regions. Moreover, in contrast to prior works where
confidence is jointly learned with flow and appearance using a single network,
we compute confidence after a warping step, and employ a separate network to
inpaint exposed regions. Empirical results on both synthetic and real datasets
show that our disentangling approach provides better occlusion maps and
produces both sharper and more realistic predictions compared to strong
baselines.

本文描述了一个用于高保真视频预测的计算模型，该模型从运动无关的生成中分离出运动特定的传播。我们引入了一个置信感知的变形算子，用于对未遮挡区域的像素预测输出进行门控，并由上下文编码器对受遮挡区域进行预测。与以往的作品不同，我们在变形步骤之后计算置信度，并使用单独的网络修补暴露区域。经验结果在合成和真实数据集上显示了我们的分离方法提供更好的遮挡映射，并与强基准相比产生更清晰，更逼真的预测。