Given a single RGB image of a complex outdoor road scene in the perspective
view, we address the novel problem of estimating an occlusion-reasoned semantic
scene layout in the top-view. This challenging problem not only requires an
accurate understanding of both the 3D geometry and the semantics of the visible
scene, but also of occluded areas. We propose a convolutional neural network
that learns to predict occluded portions of the scene layout by looking around
foreground objects like cars or pedestrians. But instead of hallucinating RGB
values, we show that directly predicting the semantics and depths in the
occluded areas enables a better transformation into the top-view. We further
show that this initial top-view representation can be significantly enhanced by
learning priors and rules about typical road layouts from simulated or, if
available, map data. Crucially, training our model does not require costly or
subjective human annotations for occluded areas or the top-view, but rather
uses readily available annotations for standard semantic segmentation. We
extensively evaluate and analyze our approach on the KITTI and Cityscapes data
sets.

研究了一个新的问题，即从复杂的室外道路场景的透视视图中估计遮挡推理的语义场景布局，提出了一种卷积神经网络方法，根据前景物体（如汽车或行人）周围的信息来预测场景布局的遮挡部分，直接预测遮挡区域的深度和语义能更好地转换成俯视图，模型的训练不需要昂贵或主观的人工标注。