In this paper, we propose a way of synthesizing realistic images directly with natural language description, which has many useful applications, e.g. intelligent image manipulation. We attempt to accomplish such synthesis: given a source image and a target text description, our model synthesizes images to meet two requirements: 1) being realistic while matching the target text description; 2) maintaining other image features that are irrelevant to the text description. The model should be able to disentangle the semantic information from the two modalities (image and text), and generate new images from the combined semantics. To achieve this, we proposed an end-to-end neural architecture that leverages adversarial learning to automatically learn implicit loss functions, which are optimized to fulfill the aforementioned two requirements. We have evaluated our model by conducting experiments on Caltech-200 bird dataset and Oxford-102 flower dataset, and have demonstrated that our model is capable of synthesizing realistic images that match the given descriptions, while still maintain other features of original images.

本文提出了一种使用自然语言描述直接合成逼真图像的方法，它有很多有用的应用，例如智能图像操作。我们提出了一种端到端的神经架构，利用对抗学习自动学习隐式损失函数，实现语义分离并生成新图像。通过在 Caltech-200 鸟类数据集和 Oxford-102 花卉数据集上进行实验，我们证明了我们的模型能够合成符合描述要求的逼真图像，同时仍保留原始图像的其他特征。

通过对抗学习进行语义图像合成