The literature on text-to-image generation is plagued by issues of faithfully
composing entities with relations. But there lacks a formal understanding of
how entity-relation compositions can be effectively learned. Moreover, the
underlying phenomenon space that meaningfully reflects the problem structure is
not well-defined, leading to an arms race for larger quantities of data in the
hope that generalization emerges out of large-scale pretraining. We hypothesize
that the underlying phenomenological coverage has not been proportionally
scaled up, leading to a skew of the presented phenomenon which harms
generalization. We introduce statistical metrics that quantify both the
linguistic and visual skew of a dataset for relational learning, and show that
generalization failures of text-to-image generation are a direct result of
incomplete or unbalanced phenomenological coverage. We first perform
experiments in a synthetic domain and demonstrate that systematically
controlled metrics are strongly predictive of generalization performance. Then
we move to natural images and show that simple distribution perturbations in
light of our theories boost generalization without enlarging the absolute data
size. This work informs an important direction towards quality-enhancing the
data diversity or balance orthogonal to scaling up the absolute size. Our
discussions point out important open questions on 1) Evaluation of generated
entity-relation compositions, and 2) Better models for reasoning with abstract
relations.

文献对文本到图像生成的研究存在实体与关系合成的问题，缺乏对有效学习实体关系组合的形式理解，以及缺乏明确定义有意义反映问题结构的现象空间，导致对大规模预训练数据期望的泛化。我们假设现象学概括的覆盖范围没有成比例地增加，导致呈现现象的偏斜对泛化有害。我们引入了统计度量模型数据集的语言和视觉偏斜，证明文本到图像生成的泛化失败是不完整或不平衡的现象学覆盖直接结果。我们首先在合成领域进行实验，证明系统控制的度量强烈预测了泛化性能。然后我们转向自然图像，并展示了在我们的理论指导下对分布进行简单扰动即可增强泛化能力，而无需扩大绝对数据规模。本研究指出了一个重要方向，即提高数据多样性或平衡，与绝对规模扩大成正交。我们的讨论指出了两个重要问题：1）评估产生的实体关系组合，以及 2）用于推理抽象关系的更好模型。