Convolutional Neural Networks (CNNs) achieve excellent computer-assisted
diagnosis with sufficient annotated training data. However, most medical
imaging datasets are small and fragmented. In this context, Generative
Adversarial Networks (GANs) can synthesize realistic/diverse additional
training images to fill the data lack in the real image distribution;
researchers have improved classification by augmenting data with noise-to-image
(e.g., random noise samples to diverse pathological images) or image-to-image
GANs (e.g., a benign image to a malignant one). Yet, no research has reported
results combining noise-to-image and image-to-image GANs for further
performance boost. Therefore, to maximize the DA effect with the GAN
combinations, we propose a two-step GAN-based DA that generates and refines
brain Magnetic Resonance (MR) images with/without tumors separately: (i)
Progressive Growing of GANs (PGGANs), multi-stage noise-to-image GAN for
high-resolution MR image generation, first generates realistic/diverse 256 X
256 images; (ii) Multimodal UNsupervised Image-to-image Translation (MUNIT)
that combines GANs/Variational AutoEncoders or SimGAN that uses a DA-focused
GAN loss, further refines the texture/shape of the PGGAN-generated images
similarly to the real ones. We thoroughly investigate CNN-based tumor
classification results, also considering the influence of pre-training on
ImageNet and discarding weird-looking GAN-generated images. The results show
that, when combined with classic DA, our two-step GAN-based DA can
significantly outperform the classic DA alone, in tumor detection (i.e.,
boosting sensitivity 93.67% to 97.48%) and also in other medical imaging tasks.

通过将噪声图像和图像到图像对抗生成网络相结合，提出了一种新的数据增强方法，能够增强磁共振样本的真实性和多样性，经过 CNN 的肿瘤检测结果表明，该方法可以显著提高经典数据增强技术的敏感度和同时对其他医学影像任务有显著提升。

结合噪声到图像 GAN 和图像到图像 GAN：用于肿瘤检测的脑 MR 图像增强

Combining Noise-to-Image and Image-to-Image GANs: Brain MR Image  Augmentation for Tumor Detection

A novel non-rigid image registration algorithm is built upon fully
convolutional networks (FCNs) to optimize and learn spatial transformations
between pairs of images to be registered in a self-supervised learning
framework. Different from most existing deep learning based image registration
methods that learn spatial transformations from training data with known
corresponding spatial transformations, our method directly estimates spatial
transformations between pairs of images by maximizing an image-wise similarity
metric between fixed and deformed moving images, similar to conventional image
registration algorithms. The image registration is implemented in a
multi-resolution image registration framework to jointly optimize and learn
spatial transformations and FCNs at different spatial resolutions with deep
self-supervision through typical feedforward and backpropagation computation.
The proposed method has been evaluated for registering 3D structural brain
magnetic resonance (MR) images and obtained better performance than
state-of-the-art image registration algorithms.

本文提出了一种新型的基于卷积神经网络的非刚性图像配准算法，通过优化和学习不同尺度下的空间变换以及卷积神经网络，实现了对 3D 结构脑磁共振图像更好的配准效果。