We present variational generative adversarial networks, a general learning
framework that combines a variational auto-encoder with a generative
adversarial network, for synthesizing images in fine-grained categories, such
as faces of a specific person or objects in a category. Our approach models an
image as a composition of label and latent attributes in a probabilistic model.
By varying the fine-grained category label fed into the resulting generative
model, we can generate images in a specific category with randomly drawn values
on a latent attribute vector. Our approach has two novel aspects. First, we
adopt a cross entropy loss for the discriminative and classifier network, but a
mean discrepancy objective for the generative network. This kind of asymmetric
loss function makes the GAN training more stable. Second, we adopt an encoder
network to learn the relationship between the latent space and the real image
space, and use pairwise feature matching to keep the structure of generated
images. We experiment with natural images of faces, flowers, and birds, and
demonstrate that the proposed models are capable of generating realistic and
diverse samples with fine-grained category labels. We further show that our
models can be applied to other tasks, such as image inpainting,
super-resolution, and data augmentation for training better face recognition
models.

本文提出一种融合自编码器和生成式对抗网络的变分生成式对抗网络模型，用于合成细粒度类别中的图像。该模型采用交叉熵损失和均值偏差目标函数，学习潜在属性和标签对图像的影响关系，并利用特征匹配保持生成图像的结构。实验结果表明该模型可以生成高度写实、具有细粒度标签的图像。