Multi-modal image fusion aims to combine information from different modes to
create a single image with comprehensive information and detailed textures.
However, fusion models based on convolutional neural networks encounter
limitations in capturing global image features due to their focus on local
convolution operations. Transformer-based models, while excelling in global
feature modeling, confront computational challenges stemming from their
quadratic complexity. Recently, the Selective Structured State Space Model has
exhibited significant potential for long-range dependency modeling with linear
complexity, offering a promising avenue to address the aforementioned dilemma.
In this paper, we propose FusionMamba, a novel dynamic feature enhancement
method for multimodal image fusion with Mamba. Specifically, we devise an
improved efficient Mamba model for image fusion, integrating efficient visual
state space model with dynamic convolution and channel attention. This refined
model not only upholds the performance of Mamba and global modeling capability
but also diminishes channel redundancy while enhancing local enhancement
capability. Additionally, we devise a dynamic feature fusion module (DFFM)
comprising two dynamic feature enhancement modules (DFEM) and a cross modality
fusion mamba module (CMFM). The former serves for dynamic texture enhancement
and dynamic difference perception, whereas the latter enhances correlation
features between modes and suppresses redundant intermodal information.
FusionMamba has yielded state-of-the-art (SOTA) performance across various
multimodal medical image fusion tasks (CT-MRI, PET-MRI, SPECT-MRI), infrared
and visible image fusion task (IR-VIS) and multimodal biomedical image fusion
dataset (GFP-PC), which is proved that our model has generalization ability.
The code for FusionMamba is available at
this https URL

提出了一种基于 FusionMamba 的动态特征增强方法，用于多模式图像融合，具有与 Mamba 相同的性能和全局建模能力，同时降低通道冗余并增强本地增强能力。证明了该模型在各种多模式医学图像融合任务（CT-MRI、PET-MRI、SPECT-MRI）、红外和可见图像融合任务（IR-VIS）以及多模式生物医学图像融合数据集（GFP-PC）中具有泛化能力。

FusionMamba: 多模态图像融合的动态特征增强

FusionMamba: Dynamic Feature Enhancement for Multimodal Image Fusion  with Mamba

Multimodal medical image fusion plays an instrumental role in several areas
of medical image processing, particularly in disease recognition and tumor
detection. Traditional fusion methods tend to process each modality
independently before combining the features and reconstructing the fusion
image. However, this approach often neglects the fundamental commonalities and
disparities between multimodal information. Furthermore, the prevailing
methodologies are largely confined to fusing two-dimensional (2D) medical image
slices, leading to a lack of contextual supervision in the fusion images and
subsequently, a decreased information yield for physicians relative to
three-dimensional (3D) images. In this study, we introduce an innovative
unsupervised feature mutual learning fusion network designed to rectify these
limitations. Our approach incorporates a Deformable Cross Feature Blend (DCFB)
module that facilitates the dual modalities in discerning their respective
similarities and differences. We have applied our model to the fusion of 3D MRI
and PET images obtained from 660 patients in the Alzheimer's Disease
Neuroimaging Initiative (ADNI) dataset. Through the application of the DCFB
module, our network generates high-quality MRI-PET fusion images. Experimental
results demonstrate that our method surpasses traditional 2D image fusion
methods in performance metrics such as Peak Signal to Noise Ratio (PSNR) and
Structural Similarity Index Measure (SSIM). Importantly, the capacity of our
method to fuse 3D images enhances the information available to physicians and
researchers, thus marking a significant step forward in the field. The code
will soon be available online.

创新的无监督特征互学习融合网络用于融合三维磁共振成像（MRI）和正电子发射断层成像（PET）图像，提高医生和研究人员的信息获取能力，并在性能度量方面优于传统二维图像融合方法。