Bias amplification is a phenomenon in which models increase imbalances
present in the training data. In this paper, we study bias amplification in the
text-to-image domain using Stable Diffusion by comparing gender ratios in
training vs. generated images. We find that the model appears to amplify
gender-occupation biases found in the training data (LAION). However, we
discover that amplification can largely be attributed to discrepancies between
training captions and model prompts. For example, an inherent difference is
that captions from the training data often contain explicit gender information
while the prompts we use do not, which leads to a distribution shift and
consequently impacts bias measures. Once we account for various distributional
differences between texts used for training and generation, we observe that
amplification decreases considerably. Our findings illustrate the challenges of
comparing biases in models and the data they are trained on, and highlight
confounding factors that contribute to bias amplification.

本文研究了文本到图像领域中的偏差放大现象，通过比较训练和生成的图像中的性别比例，发现模型似乎放大了训练数据中存在的性别职业偏见。然而，我们发现放大主要归因于训练字幕和模型提示之间的差异。当我们考虑到训练和生成所使用的文本之间的各种分布差异后，观察到放大程度显著降低。我们的研究结果说明了比较模型和训练数据中的偏见的挑战，并强调了导致偏见放大的混淆因素。