In this paper, we tackle the problem of RGB-D semantic segmentation of indoor
images. We take advantage of deconvolutional networks which can predict
pixel-wise class labels, and develop a new structure for deconvolution of
multiple modalities. We propose a novel feature transformation network to
bridge the convolutional networks and deconvolutional networks. In the feature
transformation network, we correlate the two modalities by discovering common
features between them, as well as characterize each modality by discovering
modality specific features. With the common features, we not only closely
correlate the two modalities, but also allow them to borrow features from each
other to enhance the representation of shared information. With specific
features, we capture the visual patterns that are only visible in one modality.
The proposed network achieves competitive segmentation accuracy on NYU depth
dataset V1 and V2.

本文在室内图像的 RGB-D 语义分割问题中，通过利用可以预测像素级类别标签的反卷积网络，开发了一种反卷积多模态新结构并提出了一种新的特征转换网络。该特征转换网络通过在两个模态之间发现共同特征并表征每种特性的方式将两种模态相关联。新网络在 NYU 深度数据集 V1 和 V2 上取得了有竞争力的分割精度。