The output of text-to-image synthesis systems should be coherent, clear,
photo-realistic scenes with high semantic fidelity to their conditioned text
descriptions. Our Cross-Modal Contrastive Generative Adversarial Network
(XMC-GAN) addresses this challenge by maximizing the mutual information between
image and text. It does this via multiple contrastive losses which capture
inter-modality and intra-modality correspondences. XMC-GAN uses an attentional
self-modulation generator, which enforces strong text-image correspondence, and
a contrastive discriminator, which acts as a critic as well as a feature
encoder for contrastive learning. The quality of XMC-GAN's output is a major
step up from previous models, as we show on three challenging datasets. On
MS-COCO, not only does XMC-GAN improve state-of-the-art FID from 24.70 to 9.33,
but--more importantly--people prefer XMC-GAN by 77.3 for image quality and 74.1
for image-text alignment, compared to three other recent models. XMC-GAN also
generalizes to the challenging Localized Narratives dataset (which has longer,
more detailed descriptions), improving state-of-the-art FID from 48.70 to
14.12. Lastly, we train and evaluate XMC-GAN on the challenging Open Images
data, establishing a strong benchmark FID score of 26.91.

使用交叉模态对比生成对抗网络（XMC-GAN）进行文本到图像的综合，通过使用多个对比损失，注意力自我调节生成器和对比鉴别器，以最大化图像和文本之间的相互信息，改善了图像合成的质量，并在 MS-COCO、Localized Narratives 和 Open Images 上实现了最先进的 FID 评分．

文本到图像生成的跨模态对比学习

Cross-Modal Contrastive Learning for Text-to-Image Generation

Generative Adversarial Networks (GANs) for text generation have recently
received many criticisms, as they perform worse than their MLE counterparts. We
suspect previous text GANs' inferior performance is due to the lack of a
reliable guiding signal in their discriminators. To address this problem, we
propose a generative adversarial imitation learning framework for text
generation that uses large pre-trained language models to provide more reliable
reward guidance. Our approach uses contrastive discriminator, and proximal
policy optimization (PPO) to stabilize and improve text generation performance.
For evaluation, we conduct experiments on a diverse set of unconditional and
conditional text generation tasks. Experimental results show that TextGAIL
achieves better performance in terms of both quality and diversity than the MLE
baseline. We also validate our intuition that TextGAIL's discriminator
demonstrates the capability of providing reasonable rewards with an additional
task.

本文提出一种基于预训练语言模型的文本生成生成对抗性学习框架，采用对比鉴别器和近端策略优化来稳定和改进文本生成性能，实验证明该方法在无条件和有条件文本生成任务上均优于 MLE 基线。