Semantic scene completion is the task of producing a complete 3D voxel
representation of volumetric occupancy with semantic labels for a scene from a
single-view observation. We built upon the recent work of Song et al. (CVPR
2017), who proposed SSCnet, a method that performs scene completion and
semantic labelling in a single end-to-end 3D convolutional network. SSCnet uses
only depth maps as input, even though depth maps are usually obtained from
devices that also capture colour information, such as RGBD sensors and stereo
cameras. In this work, we investigate the potential of the RGB colour channels
to improve SSCnet.

本文研究以 RGB 彩色通道提升 SSCnet 在场景补全和语义标签方面的表现

利用色彩和深度的语义场景完整性：初步实验

Semantic Scene Completion Combining Colour and Depth: preliminary  experiments

This paper focuses on semantic scene completion, a task for producing a
complete 3D voxel representation of volumetric occupancy and semantic labels
for a scene from a single-view depth map observation. Previous work has
considered scene completion and semantic labeling of depth maps separately.
However, we observe that these two problems are tightly intertwined. To
leverage the coupled nature of these two tasks, we introduce the semantic scene
completion network (SSCNet), an end-to-end 3D convolutional network that takes
a single depth image as input and simultaneously outputs occupancy and semantic
labels for all voxels in the camera view frustum. Our network uses a
dilation-based 3D context module to efficiently expand the receptive field and
enable 3D context learning. To train our network, we construct SUNCG - a
manually created large-scale dataset of synthetic 3D scenes with dense
volumetric annotations. Our experiments demonstrate that the joint model
outperforms methods addressing each task in isolation and outperforms
alternative approaches on the semantic scene completion task.

本文针对通过单视图深度图来获取场景的完整三维体素表示和语义标签的语义场景完成任务，提出了一种端到端三维卷积神经网络 SSCNet，该网络能够同时输出所有摄像机视图锥体中体素的占用和语义标签。SSCNet 使用基于扩张的三维上下文模块来高效地扩展感受野并进行三维上下文学习。实验结果表明，该联合模型比各自解决每个任务的方法表现更好，并且在语义场景完成任务上优于替代方法。