Although progress has been made for text-to-image synthesis, previous methods
fall short of generalizing to unseen or underrepresented attribute compositions
in the input text. Lacking compositionality could have severe implications for
robustness and fairness, e.g., inability to synthesize the face images of
underrepresented demographic groups. In this paper, we introduce a new
framework, StyleT2I, to improve the compositionality of text-to-image
synthesis. Specifically, we propose a CLIP-guided Contrastive Loss to better
distinguish different compositions among different sentences. To further
improve the compositionality, we design a novel Semantic Matching Loss and a
Spatial Constraint to identify attributes' latent directions for intended
spatial region manipulations, leading to better disentangled latent
representations of attributes. Based on the identified latent directions of
attributes, we propose Compositional Attribute Adjustment to adjust the latent
code, resulting in better compositionality of image synthesis. In addition, we
leverage the $\ell_2$-norm regularization of identified latent directions (norm
penalty) to strike a nice balance between image-text alignment and image
fidelity. In the experiments, we devise a new dataset split and an evaluation
metric to evaluate the compositionality of text-to-image synthesis models. The
results show that StyleT2I outperforms previous approaches in terms of the
consistency between the input text and synthesized images and achieves higher
fidelity.

本文提出了一种新的框架，StyleT2I，旨在改善文本到图像合成的组合性，并且使用 CLIP 引导的对比损失、语义匹配损失和空间约束等方法来识别属性的潜在方向，从而更好地解开属性的潜在表示，最终获得更好的合成图像的组合性。