This paper investigates an open research task of text-to-image synthesis for automatically generating or manipulating images from text descriptions. Prevailing methods mainly use the text as conditions for GAN generation, and train different models for the text-guided image generation and manipulation tasks. In this paper, we propose a novel unified framework of Cycle-consistent Inverse GAN (CI-GAN) for both text-to-image generation and text-guided image manipulation tasks. Specifically, we first train a GAN model without text input, aiming to generate images with high diversity and quality. Then we learn a GAN inversion model to convert the images back to the GAN latent space and obtain the inverted latent codes for each image, where we introduce the cycle-consistency training to learn more robust and consistent inverted latent codes. We further uncover the latent space semantics of the trained GAN model, by learning a similarity model between text representations and the latent codes. In the text-guided optimization module, we generate images with the desired semantic attributes by optimizing the inverted latent codes. Extensive experiments on the Recipe1M and CUB datasets validate the efficacy of our proposed framework.

本文提出了一种新颖的循环一致性逆向生成对抗网络(CI-GAN)框架，用于文本到图像生成和文本引导下的图像操作任务，在该框架中，我们采用循环一致性训练来学习更加稳健一致的潜变量，并通过学习文本表示与潜变量之间的相似度模型，在文本引导的优化模块中生成具有所需语义属性的图像，实验证明该框架具有良好的性能。

循环一致性反向生成对抗网络用于文本到图像合成