Multi-modality image fusion (MMIF) integrates the complementary information
from different modal images to provide comprehensive and objective
interpretation of a scenes. However, existing MMIF methods lack the ability to
resist different weather interferences in real-life scenarios, preventing them
from being useful in practical applications such as autonomous driving. To
bridge this research gap, we proposed an all-weather MMIF model. Regarding deep
learning architectures, their network designs are often viewed as a black box,
which limits their multitasking capabilities. For deweathering module, we
propose a physically-aware clear feature prediction module based on an
atmospheric scattering model that can deduce variations in light transmittance
from both scene illumination and depth. For fusion module, We utilize a
learnable low-rank representation model to decompose images into low-rank and
sparse components. This highly interpretable feature separation allows us to
better observe and understand images. Furthermore, we have established a
benchmark for MMIF research under extreme weather conditions. It encompasses
multiple scenes under three types of weather: rain, haze, and snow, with each
weather condition further subdivided into various impact levels. Extensive
fusion experiments under adverse weather demonstrate that the proposed
algorithm has excellent detail recovery and multi-modality feature extraction
capabilities.

在这篇论文中，我们提出了一种适用于各种天气条件下的多模态图像融合模型，该模型具有抵抗不同天气干扰的能力，并在极端天气条件下展示出出色的细节恢复和多模态特征提取能力。