We present a novel framework named PlaneMVS for 3D plane reconstruction from
multiple input views with known camera poses. Most previous learning-based
plane reconstruction methods reconstruct 3D planes from single images, which
highly rely on single-view regression and suffer from depth scale ambiguity. In
contrast, we reconstruct 3D planes with a multi-view-stereo (MVS) pipeline that
takes advantage of multi-view geometry. We decouple plane reconstruction into a
semantic plane detection branch and a plane MVS branch. The semantic plane
detection branch is based on a single-view plane detection framework but with
differences. The plane MVS branch adopts a set of slanted plane hypotheses to
replace conventional depth hypotheses to perform plane sweeping strategy and
finally learns pixel-level plane parameters and its planar depth map. We
present how the two branches are learned in a balanced way, and propose a
soft-pooling loss to associate the outputs of the two branches and make them
benefit from each other. Extensive experiments on various indoor datasets show
that PlaneMVS significantly outperforms state-of-the-art (SOTA) single-view
plane reconstruction methods on both plane detection and 3D geometry metrics.
Our method even outperforms a set of SOTA learning-based MVS methods thanks to
the learned plane priors. To the best of our knowledge, this is the first work
on 3D plane reconstruction within an end-to-end MVS framework.

本文提出了一种名为 PlaneMVS 的新型框架，用于从具有已知相机姿态的多个输入视图中重建 3D 平面。采用多视角几何的 MVS 流水线将平面重建解耦，采用倾斜平面假设来取代传统深度假设并最终学习像素级平面参数及其平面深度图。通过大量室内数据集的实验表明，PlaneMVS 在平面检测和 3D 几何度量上显著优于最先进的单视角平面重建方法。