Estimating the layout of a room from a single-shot panoramic image is
important in virtual/augmented reality and furniture layout simulation. This
involves identifying three-dimensional (3D) geometry, such as the location of
corners and boundaries, and performing 3D reconstruction. However, occlusion is
a common issue that can negatively impact room layout estimation, and this has
not been thoroughly studied to date. It is possible to obtain 3D shape
information of rooms as drawings of buildings and coordinates of corners from
image datasets, thus we propose providing both 2D panoramic and 3D information
to a model to effectively deal with occlusion. However, simply feeding 3D
information to a model is not sufficient to utilize the shape information for
an occluded area. Therefore, we improve the model by introducing 3D
Intersection over Union (IoU) loss to effectively use 3D information. In some
cases, drawings are not available or the construction deviates from a drawing.
Considering such practical cases, we propose a method for distilling knowledge
from a model trained with both images and 3D information to a model that takes
only images as input. The proposed model, which is called Shape-Net, achieves
state-of-the-art (SOTA) performance on benchmark datasets. We also confirmed
its effectiveness in dealing with occlusion through significantly improved
accuracy on images with occlusion compared with existing models.

该论文提出了一种名为 Shape-Net 的方法，通过将 2D 全景和 3D 信息提供给模型，引入 3D IoU 损失来解决遮挡问题，同时通过从同时利用图像和 3D 信息训练的模型中提取知识来解决实际情况下缺乏建筑图纸的问题。该模型是基于现有模型进行改良，实现了对房间布局的精确估算，同时能够有效处理具有遮挡的情况。