We propose "Conceptual Coverage Across Languages" (CoCo-CroLa), a technique
for benchmarking the degree to which any generative text-to-image system
provides multilingual parity to its training language in terms of tangible
nouns. For each model we can assess "conceptual coverage" of a given target
language relative to a source language by comparing the population of images
generated for a series of tangible nouns in the source language to the
population of images generated for each noun under translation in the target
language. This technique allows us to estimate how well-suited a model is to a
target language as well as identify model-specific weaknesses, spurious
correlations, and biases without a-priori assumptions. We demonstrate how it
can be used to benchmark T2I models in terms of multilinguality, and how
despite its simplicity it is a good proxy for impressive generalization.

这篇论文介绍了一种名为 CoCo-CroLa 的技术，用于评估任何生成式文本到图像系统在有形名词方面提供多语言的程度，可以比较源语言和目标语言中为一系列有形名词生成的图像的数量，从而估计模型对目标语言的适用性，并识别模型特定的弱点、错误关联和偏见。