In this paper, we propose a novel controllable text-to-image generative
adversarial network (ControlGAN), which can effectively synthesise high-quality
images and also control parts of the image generation according to natural
language descriptions. To achieve this, we introduce a word-level spatial and
channel-wise attention-driven generator that can disentangle different visual
attributes, and allow the model to focus on generating and manipulating
subregions corresponding to the most relevant words. Also, a word-level
discriminator is proposed to provide fine-grained supervisory feedback by
correlating words with image regions, facilitating training an effective
generator which is able to manipulate specific visual attributes without
affecting the generation of other content. Furthermore, perceptual loss is
adopted to reduce the randomness involved in the image generation, and to
encourage the generator to manipulate specific attributes required in the
modified text. Extensive experiments on benchmark datasets demonstrate that our
method outperforms existing state of the art, and is able to effectively
manipulate synthetic images using natural language descriptions. Code is
available at this https URL

该论文提出了一种新颖的可控文本到图像生成对抗网络 (ControlGAN)，它能够有效地合成高质量图像并根据自然语言描述控制图像的生成。该方法采用词级空间和通道注意力机制对不同的视觉属性进行分离，并通过词级鉴别器提供细粒度的监督反馈，使生成器能够对特定的视觉属性进行操作。通过大量的实验证明，我们的方法优于现有的最先进技术，并能够通过自然语言描述有效地操作合成图像。