RGB images differentiate from depth images as they carry more details about
the color and texture information, which can be utilized as a vital
complementary to depth for boosting the performance of 3D semantic scene
completion (SSC). SSC is composed of 3D shape completion (SC) and semantic
scene labeling while most of the existing methods use depth as the sole input
which causes the performance bottleneck. Moreover, the state-of-the-art methods
employ 3D CNNs which have cumbersome networks and tremendous parameters. We
introduce a light-weight Dimensional Decomposition Residual network (DDR) for
3D dense prediction tasks. The novel factorized convolution layer is effective
for reducing the network parameters, and the proposed multi-scale fusion
mechanism for depth and color image can improve the completion and segmentation
accuracy simultaneously. Our method demonstrates excellent performance on two
public datasets. Compared with the latest method SSCNet, we achieve 5.9% gains
in SC-IoU and 5.7% gains in SSC-IOU, albeit with only 21% network parameters
and 16.6% FLOPs employed compared with that of SSCNet.

本研究提出了一种轻量级的三维密集预测任务算法 ——Dimensional Decomposition Residual network (DDR)，该网络使用分解卷积层和多尺度融合机制来提高三维场景补全和语义分割速度和准确率，与最新的同类型算法 SSCNet 相比，我们只使用了 SSCNet 网络参数的 21%，计算量的 16.6%，但在 SC-IoU 和 SSC-IoU 上分别提升了 5.9% 和 5.7%。