In clinical practice, tri-modal medical image fusion, compared to the
existing dual-modal technique, can provide a more comprehensive view of the
lesions, aiding physicians in evaluating the disease's shape, location, and
biological activity. However, due to the limitations of imaging equipment and
considerations for patient safety, the quality of medical images is usually
limited, leading to sub-optimal fusion performance, and affecting the depth of
image analysis by the physician. Thus, there is an urgent need for a technology
that can both enhance image resolution and integrate multi-modal information.
Although current image processing methods can effectively address image fusion
and super-resolution individually, solving both problems synchronously remains
extremely challenging. In this paper, we propose TFS-Diff, a simultaneously
realize tri-modal medical image fusion and super-resolution model. Specially,
TFS-Diff is based on the diffusion model generation of a random iterative
denoising process. We also develop a simple objective function and the proposed
fusion super-resolution loss, effectively evaluates the uncertainty in the
fusion and ensures the stability of the optimization process. And the channel
attention module is proposed to effectively integrate key information from
different modalities for clinical diagnosis, avoiding information loss caused
by multiple image processing. Extensive experiments on public Harvard datasets
show that TFS-Diff significantly surpass the existing state-of-the-art methods
in both quantitative and visual evaluations. The source code will be available
at GitHub.

我们提出了一个同时实现三模态医学图像融合和超分辨率的 TFS-Diff 模型，该模型基于扩散模型生成随机迭代去噪过程，提出了简单的客观函数和融合超分辨率损失方法，通过通道注意模块有效地集成来自不同模态的关键信息，大量实验证明 TFS-Diff 在定量和视觉评估方面显著优于现有的最先进方法。

基于条件扩散模型的三模态医学图像融合和超分辨率

Simultaneous Tri-Modal Medical Image Fusion and Super-Resolution using  Conditional Diffusion Model

Image compression is a widely used technique to reduce the spatial redundancy
in images. Recently, learning based image compression has achieved significant
progress by using the powerful representation ability from neural networks.
However, the current state-of-the-art learning based image compression methods
suffer from the huge computational cost, which limits their capacity for
practical applications. In this paper, we propose a unified framework called
Efficient Deep Image Compression (EDIC) based on three new technologies,
including a channel attention module, a Gaussian mixture model and a
decoder-side enhancement module. Specifically, we design an auto-encoder style
network for learning based image compression. To improve the coding efficiency,
we exploit the channel relationship between latent representations by using the
channel attention module. Besides, the Gaussian mixture model is introduced for
the entropy model and improves the accuracy for bitrate estimation.
Furthermore, we introduce the decoder-side enhancement module to further
improve image compression performance. Our EDIC method can also be readily
incorporated with the Deep Video Compression (DVC) framework to further improve
the video compression performance. Simultaneously, our EDIC method boosts the
coding performance significantly while bringing slightly increased
computational cost. More importantly, experimental results demonstrate that the
proposed approach outperforms the current state-of-the-art image compression
methods and is up to more than 150 times faster in terms of decoding speed when
compared with Minnen's method. The proposed framework also successfully
improves the performance of the recent deep video compression system DVC. Our
code will be released at this https URL

该研究提出了一种基于三种新技术的统一框架，包括通道关注模块、高斯混合模型和解码器侧增强模块，命名为 EDIC，可用于基于学习的图像压缩，并说明该方法优于当前所有先进的图像压缩方法，同时将解码速度提高了超过 150 倍。