In clinical practice, tri-modal medical image fusion, compared to the
existing dual-modal technique, can provide a more comprehensive view of the
lesions, aiding physicians in evaluating the disease's shape, location, and
biological activity. However, due to the limitations of imaging equipment and
considerations for patient safety, the quality of medical images is usually
limited, leading to sub-optimal fusion performance, and affecting the depth of
image analysis by the physician. Thus, there is an urgent need for a technology
that can both enhance image resolution and integrate multi-modal information.
Although current image processing methods can effectively address image fusion
and super-resolution individually, solving both problems synchronously remains
extremely challenging. In this paper, we propose TFS-Diff, a simultaneously
realize tri-modal medical image fusion and super-resolution model. Specially,
TFS-Diff is based on the diffusion model generation of a random iterative
denoising process. We also develop a simple objective function and the proposed
fusion super-resolution loss, effectively evaluates the uncertainty in the
fusion and ensures the stability of the optimization process. And the channel
attention module is proposed to effectively integrate key information from
different modalities for clinical diagnosis, avoiding information loss caused
by multiple image processing. Extensive experiments on public Harvard datasets
show that TFS-Diff significantly surpass the existing state-of-the-art methods
in both quantitative and visual evaluations. The source code will be available
at GitHub.

我们提出了一个同时实现三模态医学图像融合和超分辨率的 TFS-Diff 模型，该模型基于扩散模型生成随机迭代去噪过程，提出了简单的客观函数和融合超分辨率损失方法，通过通道注意模块有效地集成来自不同模态的关键信息，大量实验证明 TFS-Diff 在定量和视觉评估方面显著优于现有的最先进方法。