Many image-to-image translation problems are ambiguous, as a single input
image may correspond to multiple possible outputs. In this work, we aim to
model a \emph{distribution} of possible outputs in a conditional generative
modeling setting. The ambiguity of the mapping is distilled in a
low-dimensional latent vector, which can be randomly sampled at test time. A
generator learns to map the given input, combined with this latent code, to the
output. We explicitly encourage the connection between output and the latent
code to be invertible. This helps prevent a many-to-one mapping from the latent
code to the output during training, also known as the problem of mode collapse,
and produces more diverse results. We explore several variants of this approach
by employing different training objectives, network architectures, and methods
of injecting the latent code. Our proposed method encourages bijective
consistency between the latent encoding and output modes. We present a
systematic comparison of our method and other variants on both perceptual
realism and diversity.

利用条件生成模型和低维潜在向量，通过鼓励输出和潜在编码间的双射一致性的方法解决图像转换中一对多的映射问题，并产生更丰富多样的结果。

多模式图像到图像翻译

Toward Multimodal Image-to-Image Translation

In this paper, we describe the "PixelGAN autoencoder", a generative
autoencoder in which the generative path is a convolutional autoregressive
neural network on pixels (PixelCNN) that is conditioned on a latent code, and
the recognition path uses a generative adversarial network (GAN) to impose a
prior distribution on the latent code. We show that different priors result in
different decompositions of information between the latent code and the
autoregressive decoder. For example, by imposing a Gaussian distribution as the
prior, we can achieve a global vs. local decomposition, or by imposing a
categorical distribution as the prior, we can disentangle the style and content
information of images in an unsupervised fashion. We further show how the
PixelGAN autoencoder with a categorical prior can be directly used in
semi-supervised settings and achieve competitive semi-supervised classification
results on the MNIST, SVHN and NORB datasets.

本文介绍了 PixelGAN 自编码器，通过条件生成对抗网络并在潜变量编码上加入不同的先验分布从而对图像信息进行全局对比或局部对比，实现了样式和内容信息的自监督分离，并在 MNIST、SVHN 和 NORB 数据集上实现了竞争力的半监督分类结果。