The deficiency of 3D segmentation labels is one of the main obstacles to
effective point cloud segmentation, especially for scenes in the wild with
varieties of different objects. To alleviate this issue, we propose a novel
deep graph convolutional network-based framework for large-scale semantic scene
segmentation in point clouds with sole 2D supervision. Different with numerous
preceding multi-view supervised approaches focusing on single object point
clouds, we argue that 2D supervision is capable of providing sufficient
guidance information for training 3D semantic segmentation models of natural
scene point clouds while not explicitly capturing their inherent structures,
even with only single view per training sample. Specifically, a Graph-based
Pyramid Feature Network (GPFN) is designed to implicitly infer both global and
local features of point sets and an Observability Network (OBSNet) is
introduced to further solve object occlusion problem caused by complicated
spatial relations of objects in 3D scenes. During the projection process,
perspective rendering and semantic fusion modules are proposed to provide
refined 2D supervision signals for training along with a 2D-3D joint
optimization strategy. Extensive experimental results demonstrate the
effectiveness of our 2D supervised framework, which achieves comparable results
with the state-of-the-art approaches trained with full 3D labels, for semantic
point cloud segmentation on the popular SUNCG synthetic dataset and S3DIS
real-world dataset.

提出了一种基于 2D 监督的新型深度图卷积网络框架，用于大规模语义场景点云的分割，通过 Graph-based Pyramid Feature Network 和 Observability Network 可以隐式推断点集的全局和局部特征，并提出了一个 2D-3D 联合优化策略用于优化投影过程中的语义信息的加入， 即使只有单个训练样本的情况下，也能够进行自然场景下的 3D 分割训练，扩展实验在 SUNCG 和 S3DIS 数据集上证明了这种 2D 监督框架在语义点云分割方面的有效性。