Generating virtual populations (VPs) of anatomy is essential for conducting
in-silico trials of medical devices. Typically, the generated VP should capture
sufficient variability while remaining plausible, and should reflect specific
characteristics and patient demographics observed in real populations. It is
desirable in several applications to synthesize VPs in a \textit{controlled}
manner, where relevant covariates are used to conditionally synthesise virtual
populations that fit specific target patient populations/characteristics. We
propose to equip a conditional variational autoencoder (cVAE) with normalizing
flows to boost the flexibility and complexity of the approximate posterior
learned, leading to enhanced flexibility for controllable synthesis of VPs of
anatomical structures. We demonstrate the performance of our conditional-flow
VAE using a dataset of cardiac left ventricles acquired from 2360 patients,
with associated demographic information and clinical measurements (used as
covariates/conditioning information). The obtained results indicate the
superiority of the proposed method for conditional synthesis of virtual
populations of cardiac left ventricles relative to a cVAE. Conditional
synthesis performance was assessed in terms of generalisation and specificity
errors, and in terms of the ability to preserve clinical relevant biomarkers in
the synthesised VPs, I.e. left ventricular blood pool and myocardial volume,
relative to the observed real population.

使用带有规范化流的条件变分自编码器 (cVAE) 来进行大量人体结构的虚拟人群条件合成，在临床应用中优越表现。

一种用于可控解剖虚拟人群合成的条件流变分自动编码器

A Conditional Flow Variational Autoencoder for Controllable Synthesis of  Virtual Populations of Anatomy

Recently most successful image synthesis models are multi stage process to
combine the advantages of different methods, which always includes a VAE-like
model for faithfully reconstructing embedding to image and a prior model to
generate image embedding. At the same time, diffusion models have shown be
capacity to generate high-quality synthetic images. Our work proposes a VQ-VAE
architecture model with a diffusion decoder (DiVAE) to work as the
reconstructing component in image synthesis. We explore how to input image
embedding into diffusion model for excellent performance and find that simple
modification on diffusion's UNet can achieve it. Training on ImageNet, Our
model achieves state-of-the-art results and generates more photorealistic
images specifically. In addition, we apply the DiVAE with an Auto-regressive
generator on conditional synthesis tasks to perform more human-feeling and
detailed samples.

该研究提出了一种使用 VQ-VAE 和扩散解码器的 DiVAE 图像生成模型，相关实验结果表明该模型能够生成高度逼真的图像，特别适用于条件综合任务。

DiVAE：具有去噪扩散解码器的真实感图像合成

DiVAE: Photorealistic Images Synthesis with Denoising Diffusion Decoder

Designed to learn long-range interactions on sequential data, transformers
continue to show state-of-the-art results on a wide variety of tasks. In
contrast to CNNs, they contain no inductive bias that prioritizes local
interactions. This makes them expressive, but also computationally infeasible
for long sequences, such as high-resolution images. We demonstrate how
combining the effectiveness of the inductive bias of CNNs with the expressivity
of transformers enables them to model and thereby synthesize high-resolution
images. We show how to (i) use CNNs to learn a context-rich vocabulary of image
constituents, and in turn (ii) utilize transformers to efficiently model their
composition within high-resolution images. Our approach is readily applied to
conditional synthesis tasks, where both non-spatial information, such as object
classes, and spatial information, such as segmentations, can control the
generated image. In particular, we present the first results on
semantically-guided synthesis of megapixel images with transformers and obtain
the state of the art among autoregressive models on class-conditional ImageNet.
Code and pretrained models can be found at
this https URL .

结合卷积神经网络与 transformers 模型的特点，实现了生成高分辨率图像的任务，并取得了 ImageNet 中类别有条件下的自回归模型的最优结果

驯服 Transformer 的高分辨率图像合成

Taming Transformers for High-Resolution Image Synthesis

Recent improvements in generative adversarial visual synthesis incorporate
real and fake image transformation in a self-supervised setting, leading to
increased stability and perceptual fidelity. However, these approaches
typically involve image augmentations via additional regularizers in the GAN
objective and thus spend valuable network capacity towards approximating
transformation equivariance instead of their desired task. In this work, we
explicitly incorporate inductive symmetry priors into the network architectures
via group-equivariant convolutional networks. Group-convolutions have higher
expressive power with fewer samples and lead to better gradient feedback
between generator and discriminator. We show that group-equivariance integrates
seamlessly with recent techniques for GAN training across regularizers,
architectures, and loss functions. We demonstrate the utility of our methods
for conditional synthesis by improving generation in the limited data regime
across symmetric imaging datasets and even find benefits for natural images
with preferred orientation.

该研究采用群等变卷积网络采用显式诱导对称性先验，有效提高生成对抗网络中条件综合的表现，在对称成像数据集方面表现尤为显著。

群等变生成对抗网络

Group Equivariant Generative Adversarial Networks

We present a simple nearest-neighbor (NN) approach that synthesizes
high-frequency photorealistic images from an "incomplete" signal such as a
low-resolution image, a surface normal map, or edges. Current state-of-the-art
deep generative models designed for such conditional image synthesis lack two
important things: (1) they are unable to generate a large set of diverse
outputs, due to the mode collapse problem. (2) they are not interpretable,
making it difficult to control the synthesized output. We demonstrate that NN
approaches potentially address such limitations, but suffer in accuracy on
small datasets. We design a simple pipeline that combines the best of both
worlds: the first stage uses a convolutional neural network (CNN) to maps the
input to a (overly-smoothed) image, and the second stage uses a pixel-wise
nearest neighbor method to map the smoothed output to multiple high-quality,
high-frequency outputs in a controllable manner. We demonstrate our approach
for various input modalities, and for various domains ranging from human faces
to cats-and-dogs to shoes and handbags.

本文提出了一种简单的最近邻方法，可以从 “不完整” 的信号（如低分辨率图像、表面法线图或边缘）中合成高频逼真的图像。通过将卷积神经网络和最邻近法相结合，可以解决模式崩溃问题和输出可控性。