We present a deep learning framework, called DuLa-Net, to predict
Manhattan-world 3D room layouts from a single RGB panorama. To achieve better
prediction accuracy, our method leverages two projections of the panorama at
once, namely the equirectangular panorama-view and the perspective
ceiling-view, that each contains different clues about the room layouts. Our
network architecture consists of two encoder-decoder branches for analyzing
each of the two views. In addition, a novel feature fusion structure is
proposed to connect the two branches, which are then jointly trained to predict
the 2D floor plans and layout heights. To learn more complex room layouts, we
introduce the Realtor360 dataset that contains panoramas of Manhattan-world
room layouts with different numbers of corners. Experimental results show that
our work outperforms recent state-of-the-art in prediction accuracy and
performance, especially in the rooms with non-cuboid layouts.

本篇研究论文介绍了一个名为 DuLa-Net 的深度学习框架，可从单个 RGB 全景图中预测曼哈顿世界 3D 房间布局，该方法利用全景视图和俯视图的两个投影来提高预测精度，以及一个新颖的特征融合结构用于连接这两个分支，并进行联合训练来预测 2D 楼层平面图和布局高度，实验结果表明，在预测精度和性能方面，尤其是具有非长方体布局的房间，我们的方法优于最新的最先进技术。同时我们引入了 Realtor360 数据集，其中包含了曼哈顿世界房间布局的全景图像。