We revisit Semantic Scene Completion (SSC), a useful task to predict the
semantic and occupancy representation of 3D scenes, in this paper. A number of
methods for this task are always based on voxelized scene representations for
keeping local scene structure. However, due to the existence of visible empty
voxels, these methods always suffer from heavy computation redundancy when the
network goes deeper, and thus limit the completion quality. To address this
dilemma, we propose our novel point-voxel aggregation network for this task.
Firstly, we transfer the voxelized scenes to point clouds by removing these
visible empty voxels and adopt a deep point stream to capture semantic
information from the scene efficiently. Meanwhile, a light-weight voxel stream
containing only two 3D convolution layers preserves local structures of the
voxelized scenes. Furthermore, we design an anisotropic voxel aggregation
operator to fuse the structure details from the voxel stream into the point
stream, and a semantic-aware propagation module to enhance the up-sampling
process in the point stream by semantic labels. We demonstrate that our model
surpasses state-of-the-arts on two benchmarks by a large margin, with only
depth images as the input.

本文介绍了一种名为 Semantic Scene Completion 的任务，提出了一个使用点和体素来预测 3D 场景语义和占用表示的点 - 体素聚合网络，该网络使用深度点流来高效捕捉场景中的语义信息，以及只含两个 3D 卷积层的轻量化体素流来保持场景的局部结构。通过使用各向异性体素聚合操作融合体素流中的结构细节，以及使用基于语义标签的传播模块来增强点流中的上采样过程，该模型在两个基准测试中超过了当前最先进技术。