RGB thermal scene parsing has recently attracted increasing research interest
in the field of computer vision. However, most existing methods fail to perform
good boundary extraction for prediction maps and cannot fully use high level
features. In addition, these methods simply fuse the features from RGB and
thermal modalities but are unable to obtain comprehensive fused features. To
address these problems, we propose an edge-aware guidance fusion network
(EGFNet) for RGB thermal scene parsing. First, we introduce a prior edge map
generated using the RGB and thermal images to capture detailed information in
the prediction map and then embed the prior edge information in the feature
maps. To effectively fuse the RGB and thermal information, we propose a
multimodal fusion module that guarantees adequate cross-modal fusion.
Considering the importance of high level semantic information, we propose a
global information module and a semantic information module to extract rich
semantic information from the high-level features. For decoding, we use simple
elementwise addition for cascaded feature fusion. Finally, to improve the
parsing accuracy, we apply multitask deep supervision to the semantic and
boundary maps. Extensive experiments were performed on benchmark datasets to
demonstrate the effectiveness of the proposed EGFNet and its superior
performance compared with state of the art methods. The code and results can be
found at this https URL

提出一种边缘感知融合引导网络（EGFNet）用于 RGB 热场景解析，在 RGB 和热图像上生成先验边缘映射捕捉详细信息，使用多模态融合模块来融合 RGB 和热信息，使用全局信息和语义信息模块提取高层次特征的语义信息，应用多任务深度监督来提高解析准确性，实验结果表明 EGFNet 是优于现有方法的。