As a voxel-wise labeling task, semantic scene completion (SSC) tries to
simultaneously infer the occupancy and semantic labels for a scene from a
single depth and/or RGB image. The key challenge for SSC is how to effectively
take advantage of the 3D context to model various objects or stuffs with severe
variations in shapes, layouts and visibility. To handle such variations, we
propose a novel module called anisotropic convolution, which properties with
flexibility and power impossible for the competing methods such as standard 3D
convolution and some of its variations. In contrast to the standard 3D
convolution that is limited to a fixed 3D receptive field, our module is
capable of modeling the dimensional anisotropy voxel-wisely. The basic idea is
to enable anisotropic 3D receptive field by decomposing a 3D convolution into
three consecutive 1D convolutions, and the kernel size for each such 1D
convolution is adaptively determined on the fly. By stacking multiple such
anisotropic convolution modules, the voxel-wise modeling capability can be
further enhanced while maintaining a controllable amount of model parameters.
Extensive experiments on two SSC benchmarks, NYU-Depth-v2 and NYUCAD, show the
superior performance of the proposed method. Our code is available at
this https URL

提出了一种名为各向异性卷积的新模块，用于解决语义场景补全任务中物体或材料形状、布局和可见性变化的挑战，该模块能够在体素上进行维度各向异性的建模，通过在多个各向异性卷积模块中堆叠，进一步增强了体素模型的建模能力，在两个语义场景补全基准测试上表现优越。