Infrared and visible image fusion is a powerful technique that combines
complementary information from different modalities for downstream semantic
perception tasks. Existing learning-based methods show remarkable performance,
but are suffering from the inherent vulnerability of adversarial attacks,
causing a significant decrease in accuracy. In this work, a perception-aware
fusion framework is proposed to promote segmentation robustness in adversarial
scenes. We first conduct systematic analyses about the components of image
fusion, investigating the correlation with segmentation robustness under
adversarial perturbations. Based on these analyses, we propose a harmonized
architecture search with a decomposition-based structure to balance standard
accuracy and robustness. We also propose an adaptive learning strategy to
improve the parameter robustness of image fusion, which can learn effective
feature extraction under diverse adversarial perturbations. Thus, the goals of
image fusion (\textit{i.e.,} extracting complementary features from source
modalities and defending attack) can be realized from the perspectives of
architectural and learning strategies. Extensive experimental results
demonstrate that our scheme substantially enhances the robustness, with gains
of 15.3% mIOU of segmentation in the adversarial scene, compared with advanced
competitors. The source codes are available at
this https URL

提出了一种感知感知融合框架来提高对抗环境中的分割鲁棒性，通过系统分析图像融合的组成部分与对抗扰动下的分割鲁棒性的相关性，并提出一种基于分解结构的协同设计方法来平衡标准准确率和鲁棒性，以及一种自适应学习策略来提高图像融合的参数鲁棒性。实验结果表明，与先进竞争对手相比，在对抗场景中分割的 mIOU 提高了 15.3%。