Text-to-image diffusion models can synthesize a large variety of concepts in
new compositions and scenarios. However, they still struggle with generating
uncommon concepts, rare unusual combinations, or structured concepts like hand
palms. Their limitation is partly due to the long-tail nature of their training
data: web-crawled data sets are strongly unbalanced, causing models to
under-represent concepts from the tail of the distribution. Here we
characterize the effect of unbalanced training data on text-to-image models and
offer a remedy. We show that rare concepts can be correctly generated by
carefully selecting suitable generation seeds in the noise space, a technique
that we call SeedSelect. SeedSelect is efficient and does not require
retraining the diffusion model. We evaluate the benefit of SeedSelect on a
series of problems. First, in few-shot semantic data augmentation, where we
generate semantically correct images for few-shot and long-tail benchmarks. We
show classification improvement on all classes, both from the head and tail of
the training data of diffusion models. We further evaluate SeedSelect on
correcting images of hands, a well-known pitfall of current diffusion models,
and show that it improves hand generation substantially.

研究了长尾训练数据对于文本到图像模型的影响，并提出了 SeedSelect 技术，通过在噪声空间中选择适当的初始种子，可以有效地生成罕见的图像概念，提高了模型效果，特别是在手部图像生成方面的表现得到了明显改善。