Synthesizing high-quality photorealistic images with textual descriptions as
a condition is very challenging. Generative Adversarial Networks (GANs), the
classical model for this task, frequently suffer from low consistency between
image and text descriptions and insufficient richness in synthesized images.
Recently, conditional affine transformations (CAT), such as conditional batch
normalization and instance normalization, have been applied to different layers
of GAN to control content synthesis in images. CAT is a multi-layer perceptron
that independently predicts data based on batch statistics between neighboring
layers, with global textual information unavailable to other layers. To address
this issue, we first model CAT and a recurrent neural network (RAT) to ensure
that different layers can access global information. We then introduce shuffle
attention between RAT to mitigate the characteristic of information forgetting
in recurrent neural networks. Moreover, both our generator and discriminator
utilize the powerful pre-trained model, Clip, which has been extensively
employed for establishing associations between text and images through the
learning of multimodal representations in latent space. The discriminator
utilizes CLIP's ability to comprehend complex scenes to accurately assess the
quality of the generated images. Extensive experiments have been conducted on
the CUB, Oxford, and CelebA-tiny datasets to demonstrate the superiority of the
proposed model over current state-of-the-art models. The code is
this https URL

通过使用条件仿射变换以及在循环神经网络之间引入随机注意力机制，结合强大的预训练模型 Clip，本研究提出了一种高质量逼真图像的生成模型，有效提升了图像与文本描述之间的一致性和综合图像的丰富度。

RATLIP: 基于递归仿射变换的生成对抗网络 CLIP 文本到图像合成

RATLIP: Generative Adversarial CLIP Text-to-Image Synthesis Based on  Recurrent Affine Transformations

Attention mechanisms, which enable a neural network to accurately focus on
all the relevant elements of the input, have become an essential component to
improve the performance of deep neural networks. There are mainly two attention
mechanisms widely used in computer vision studies, \textit{spatial attention}
and \textit{channel attention}, which aim to capture the pixel-level pairwise
relationship and channel dependency, respectively. Although fusing them
together may achieve better performance than their individual implementations,
it will inevitably increase the computational overhead. In this paper, we
propose an efficient Shuffle Attention (SA) module to address this issue, which
adopts Shuffle Units to combine two types of attention mechanisms effectively.
Specifically, SA first groups channel dimensions into multiple sub-features
before processing them in parallel. Then, for each sub-feature, SA utilizes a
Shuffle Unit to depict feature dependencies in both spatial and channel
dimensions. After that, all sub-features are aggregated and a "channel shuffle"
operator is adopted to enable information communication between different
sub-features. The proposed SA module is efficient yet effective, e.g., the
parameters and computations of SA against the backbone ResNet50 are 300 vs.
25.56M and 2.76e-3 GFLOPs vs. 4.12 GFLOPs, respectively, and the performance
boost is more than 1.34% in terms of Top-1 accuracy. Extensive experimental
results on common-used benchmarks, including ImageNet-1k for classification, MS
COCO for object detection, and instance segmentation, demonstrate that the
proposed SA outperforms the current SOTA methods significantly by achieving
higher accuracy while having lower model complexity. The code and models are
available at this https URL

该论文提出了一种名为 Shuffle Attention（SA）的机制，通过使用 Shuffle Unit 来同时描述空间和通道维度中的特征依赖关系，以有效地结合两种注意力机制。实验结果表明，该机制比当前 SOTA 方法具有更高的准确性，同时具有更低的模型复杂度。