Multi-modal medical images provide complementary soft-tissue characteristics
that aid in the screening and diagnosis of diseases. However, limited scanning
time, image corruption and various imaging protocols often result in incomplete
multi-modal images, thus limiting the usage of multi-modal data for clinical
purposes. To address this issue, in this paper, we propose a novel unified
multi-modal image synthesis method for missing modality imputation. Our method
overall takes a generative adversarial architecture, which aims to synthesize
missing modalities from any combination of available ones with a single model.
To this end, we specifically design a Commonality- and Discrepancy-Sensitive
Encoder for the generator to exploit both modality-invariant and specific
information contained in input modalities. The incorporation of both types of
information facilitates the generation of images with consistent anatomy and
realistic details of the desired distribution. Besides, we propose a Dynamic
Feature Unification Module to integrate information from a varying number of
available modalities, which enables the network to be robust to random missing
modalities. The module performs both hard integration and soft integration,
ensuring the effectiveness of feature combination while avoiding information
loss. Verified on two public multi-modal magnetic resonance datasets, the
proposed method is effective in handling various synthesis tasks and shows
superior performance compared to previous methods.

该论文提出了一种新的统一的多模式图像综合方法，包括通过生成敌对网络从任意可用模式的组合中合成丢失的模态，并使用共性和差异敏感编码器及动态特征统一模块来提高图像合成质量和适应性，实验结果表明该方法在处理多种综合任务时具有优越的性能。

缺失模态插值的统一多模态图像合成

Unified Multi-Modal Image Synthesis for Missing Modality Imputation

This work presents a generative adversarial architecture for generating
three-dimensional shapes based on signed distance representations. While the
deep generation of shapes has been mostly tackled by voxel and surface point
cloud approaches, our generator learns to approximate the signed distance for
any point in space given prior latent information. Although structurally
similar to generative point cloud approaches, this formulation can be evaluated
with arbitrary point density during inference, leading to fine-grained details
in generated outputs. Furthermore, we study the effects of using either
progressively growing voxel- or point-processing networks as discriminators,
and propose a refinement scheme to strengthen the generator's capabilities in
modeling the zero iso-surface decision boundary of shapes. We train our
approach on the ShapeNet benchmark dataset and validate, both quantitatively
and qualitatively, its performance in generating realistic 3D shapes.

本研究提出了一种基于有符号距离表示的生成对抗神经网络架构，能够生成高保真度的三维模型，并通过使用不同的 progressively growing voxel 网络作为鉴别器以及细化生成器能力的算法，从 ShapeNet 基准数据集中验证了其有效性。