Semantic Scene Completion (SSC) aims to simultaneously predict the volumetric
occupancy and semantic category of a 3D scene. It helps intelligent devices to
understand and interact with the surrounding scenes. Due to the high-memory
requirement, current methods only produce low-resolution completion
predictions, and generally lose the object details. Furthermore, they also
ignore the multi-scale spatial contexts, which play a vital role for the 3D
inference. To address these issues, in this work we propose a novel deep
learning framework, named Cascaded Context Pyramid Network (CCPNet), to jointly
infer the occupancy and semantic labels of a volumetric 3D scene from a single
depth image. The proposed CCPNet improves the labeling coherence with a
cascaded context pyramid. Meanwhile, based on the low-level features, it
progressively restores the fine-structures of objects with Guided Residual
Refinement (GRR) modules. Our proposed framework has three outstanding
advantages: (1) it explicitly models the 3D spatial context for performance
improvement; (2) full-resolution 3D volumes are produced with
structure-preserving details; (3) light-weight models with low-memory
requirements are captured with a good extensibility. Extensive experiments
demonstrate that in spite of taking a single-view depth map, our proposed
framework can generate high-quality SSC results, and outperforms
state-of-the-art approaches on both the synthetic SUNCG and real NYU datasets.

本研究提出了一种新的深度学习框架 CCNPNet，它采用级联金字塔结构模型和引导型残差细化模块，可从单张深度图像中推断出体素化 3D 场景的占用率和语义标签，并在 SUNCG 和 NYU 数据集上取得了优于现有方法的高质量 SSC 结果。