Semantic Scene Completion (SSC) aims to jointly infer semantics and
occupancies of 3D scenes. Truncated Signed Distance Function (TSDF), a 3D
encoding of depth, has been a common input for SSC. Furthermore, RGB-TSDF
fusion, seems promising since these two modalities provide color and geometry
information, respectively. Nevertheless, RGB-TSDF fusion has been considered
nontrivial and commonly-used naive addition will result in inconsistent
results. We argue that the inconsistency comes from the sparsity of RGB
features upon projecting into 3D space, while TSDF features are dense, leading
to imbalanced feature maps when summed up. To address this RGB-TSDF
distribution difference, we propose a two-stage network with a 3D RGB feature
completion module that completes RGB features with meaningful values for
occluded areas. Moreover, we propose an effective classwise entropy loss
function to punish inconsistency. Extensive experiments on public datasets
verify that our method achieves state-of-the-art performance among methods that
do not adopt extra data.

基于 RGB-TSDF 融合的 3D 场景补全方法中存在局部密度差异的问题以及不一致性，本文提出了一个两阶段网络，包括一个 3D RGB 特征补全模块用来处理 RGB 特征在 3D 空间中的稀疏性问题，并且引入了一个有效的类内熵损失函数来惩罚不一致性，实验证明本文方法在不使用额外数据的情况下可以获得最先进的性能。