In this work we present a method to train a plane-aware convolutional neural
network for dense depth and surface normal estimation as well as plane
boundaries from a single indoor $360^\circ$ image. Using our proposed loss
function, our network outperforms existing methods for single-view, indoor,
omnidirectional depth estimation and provides an initial benchmark for surface
normal prediction from $360^\circ$ images. Our improvements are due to the use
of a novel plane-aware loss that leverages principal curvature as an indicator
of planar boundaries. We also show that including geodesic coordinate maps as
network priors provides a significant boost in surface normal prediction
accuracy. Finally, we demonstrate how we can combine our network's outputs to
generate high quality 3D "pop-up" models of indoor scenes.

本研究提出了一种针对单个室内 360 度图像的平面感知卷积神经网络方法，可实现密集的深度和表面法向估计以及平面边界识别，此外，添加测地线坐标图作为网络先验可显著提高表面法向预测准确性，最终可以将网络的输出组合用于生成高质量的室内场景 3D 模型。

平面感知网络室内三维重建

Pano Popups: Indoor 3D Reconstruction with a Plane-Aware Network

In this work, we introduce the novel problem of identifying dense canonical
3D coordinate frames from a single RGB image. We observe that each pixel in an
image corresponds to a surface in the underlying 3D geometry, where a canonical
frame can be identified as represented by three orthogonal axes, one along its
normal direction and two in its tangent plane. We propose an algorithm to
predict these axes from RGB. Our first insight is that canonical frames
computed automatically with recently introduced direction field synthesis
methods can provide training data for the task. Our second insight is that
networks designed for surface normal prediction provide better results when
trained jointly to predict canonical frames, and even better when trained to
also predict 2D projections of canonical frames. We conjecture this is because
projections of canonical tangent directions often align with local gradients in
images, and because those directions are tightly linked to 3D canonical frames
through projective geometry and orthogonality constraints. In our experiments,
we find that our method predicts 3D canonical frames that can be used in
applications ranging from surface normal estimation, feature matching, and
augmented reality.

本文介绍一种从单个 RGB 图像中识别密集规范三维坐标系的方法，提出了一种算法来预测这些坐标轴，实验证明我们的方法可以应用于表面法线估计、特征匹配和增强现实等应用。