Semantic segmentation of RGB-D images involves understanding the appearance
and spatial relationships of objects within a scene, which requires careful
consideration of various factors. However, in indoor environments, the simple
input of RGB and depth images often results in a relatively limited acquisition
of semantic and spatial information, leading to suboptimal segmentation
outcomes. To address this, we propose the Multi-modal Interaction and Pooling
Attention Network (MIPANet), a novel approach designed to harness the
interactive synergy between RGB and depth modalities, optimizing the
utilization of complementary information. Specifically, we incorporate a
Multi-modal Interaction Fusion Module (MIM) into the deepest layers of the
network. This module is engineered to facilitate the fusion of RGB and depth
information, allowing for mutual enhancement and correction. Additionally, we
introduce a Pooling Attention Module (PAM) at various stages of the encoder.
This module serves to amplify the features extracted by the network and
integrates the module's output into the decoder in a targeted manner,
significantly improving semantic segmentation performance. Our experimental
results demonstrate that MIPANet outperforms existing methods on two indoor
scene datasets, NYUDv2 and SUN-RGBD, underscoring its effectiveness in
enhancing RGB-D semantic segmentation.

提出了一种利用 RGB 和深度模态之间互动协同作用的多模态交互与池化注意网络（MIPANet）来优化室内场景中 RGB-D 语义分割的方法，通过多模态交互与融合模块 (MIM) 和池化注意模块 (PAM) 的引入，有效改善了 RGB-D 语义分割性能。