While conventional depth estimation can infer the geometry of a scene from a
single RGB image, it fails to estimate scene regions that are occluded by
foreground objects. This limits the use of depth prediction in augmented and
virtual reality applications, that aim at scene exploration by synthesizing the
scene from a different vantage point, or at diminished reality. To address this
issue, we shift the focus from conventional depth map prediction to the
regression of a specific data representation called Layered Depth Image (LDI),
which contains information about the occluded regions in the reference frame
and can fill in occlusion gaps in case of small view changes. We propose a
novel approach based on Convolutional Neural Networks (CNNs) to jointly predict
depth maps and foreground separation masks used to condition Generative
Adversarial Networks (GANs) for hallucinating plausible color and depths in the
initially occluded areas. We demonstrate the effectiveness of our approach for
novel scene view synthesis from a single image.

该论文提出了一种基于卷积神经网络的方法，通过预测包含被前景遮挡区域信息的分层深度图像和前景分离掩模来合成场景和填补遮挡的空白，用于虚拟现实和减弱现实领域的单幅图像的新视点合成。