MonoScene proposes a 3D Semantic Scene Completion (SSC) framework, where the
dense geometry and semantics of a scene are inferred from a single monocular
RGB image. Different from the SSC literature, relying on 2.5 or 3D input, we
solve the complex problem of 2D to 3D scene reconstruction while jointly
inferring its semantics. Our framework relies on successive 2D and 3D UNets
bridged by a novel 2D-3D features projection inspiring from optics and
introduces a 3D context relation prior to enforce spatio-semantic consistency.
Along with architectural contributions, we introduce novel global scene and
local frustums losses. Experiments show we outperform the literature on all
metrics and datasets while hallucinating plausible scenery even beyond the
camera field of view. Our code and trained models are available at
this https URL

MonoScene 提出了一种 3D 语义场景完成（SSC）框架，该框架从单个单眼 RGB 图像推断出场景的密集几何和语义，在 2D 和 3D UNets 的连续 2D 和 3D 特征投影的基础上引入了 3D 上下文关系先验以强制实现空间 - 语义一致性。与先前的研究不同的是，该框架在联合推断其语义的同时解决了 2D 到 3D 的场景重建问题。实验表明，我们在所有指标和数据集上均优于先前的研究，甚至能够虚构出超出镜头视野的逼真场景。