Multi-modal medical images provide complementary soft-tissue characteristics
that aid in the screening and diagnosis of diseases. However, limited scanning
time, image corruption and various imaging protocols often result in incomplete
multi-modal images, thus limiting the usage of multi-modal data for clinical
purposes. To address this issue, in this paper, we propose a novel unified
multi-modal image synthesis method for missing modality imputation. Our method
overall takes a generative adversarial architecture, which aims to synthesize
missing modalities from any combination of available ones with a single model.
To this end, we specifically design a Commonality- and Discrepancy-Sensitive
Encoder for the generator to exploit both modality-invariant and specific
information contained in input modalities. The incorporation of both types of
information facilitates the generation of images with consistent anatomy and
realistic details of the desired distribution. Besides, we propose a Dynamic
Feature Unification Module to integrate information from a varying number of
available modalities, which enables the network to be robust to random missing
modalities. The module performs both hard integration and soft integration,
ensuring the effectiveness of feature combination while avoiding information
loss. Verified on two public multi-modal magnetic resonance datasets, the
proposed method is effective in handling various synthesis tasks and shows
superior performance compared to previous methods.

该论文提出了一种新的统一的多模式图像综合方法，包括通过生成敌对网络从任意可用模式的组合中合成丢失的模态，并使用共性和差异敏感编码器及动态特征统一模块来提高图像合成质量和适应性，实验结果表明该方法在处理多种综合任务时具有优越的性能。