Compared to RGB semantic segmentation, RGBD semantic segmentation can achieve
better performance by taking depth information into consideration. However, it
is still problematic for contemporary segmenters to effectively exploit RGBD
information since the feature distributions of RGB and depth (D) images vary
significantly in different scenes. In this paper, we propose an Attention
Complementary Network (ACNet) that selectively gathers features from RGB and
depth branches. The main contributions lie in the Attention Complementary
Module (ACM) and the architecture with three parallel branches. More precisely,
ACM is a channel attention-based module that extracts weighted features from
RGB and depth branches. The architecture preserves the inference of the
original RGB and depth branches, and enables the fusion branch at the same
time. Based on the above structures, ACNet is capable of exploiting more
high-quality features from different channels. We evaluate our model on
SUN-RGBD and NYUDv2 datasets, and prove that our model outperforms
state-of-the-art methods. In particular, a mIoU score of 48.3\% on NYUDv2 test
set is achieved with ResNet50. We will release our source code based on PyTorch
and the trained segmentation model at this https URL

本文提出了一种新的方法 ACNet，通过 Attention Complementary Module (ACM) 和 三个并行分支的结构，ACNet 能够从不同通道提取更多高质量的特征，在 SUN-RGBD 和 NYUDv2 数据集上得到了更好的表现，特别地，在 NYUDv2 测试集上，ResNet50 取得了 48.3% 的 mIoU 得分，并在 https URL 上发布了源代码和经过训练的分割模型。