Infrared and visible image fusion (IVF) plays an important role in
intelligent transportation system (ITS). The early works predominantly focus on
boosting the visual appeal of the fused result, and only several recent
approaches have tried to combine the high-level vision task with IVF. However,
they prioritize the design of cascaded structure to seek unified suitable
features and fit different tasks. Thus, they tend to typically bias toward to
reconstructing raw pixels without considering the significance of semantic
features. Therefore, we propose a novel prior semantic guided image fusion
method based on the dual-modality strategy, improving the performance of IVF in
ITS. Specifically, to explore the independent significant semantic of each
modality, we first design two parallel semantic segmentation branches with a
refined feature adaptive-modulation (RFaM) mechanism. RFaM can perceive the
features that are semantically distinct enough in each semantic segmentation
branch. Then, two pilot experiments based on the two branches are conducted to
capture the significant prior semantic of two images, which then is applied to
guide the fusion task in the integration of semantic segmentation branches and
fusion branches. In addition, to aggregate both high-level semantics and
impressive visual effects, we further investigate the frequency response of the
prior semantics, and propose a multi-level representation-adaptive fusion
(MRaF) module to explicitly integrate the low-frequent prior semantic with the
high-frequent details. Extensive experiments on two public datasets demonstrate
the superiority of our method over the state-of-the-art image fusion
approaches, in terms of either the visual appeal or the high-level semantics.

基于双模性策略的先验语义引导图像融合方法提供了在智能交通系统中提高红外和可见光图像融合性能的新途径，通过设计并应用两个并行的语义分割支路和一种自适应调制机制，捕捉和整合了两个图像的重要的先验语义信息，并通过多级表示自适应融合模块进一步整合高层语义和细节，优于现有图像融合方法。