The size of pretrained models is increasing, and so is their performance on a variety of NLP tasks. However, as their memorization capacity grows, they might pick up more social biases. In this work, we examine the connection between model size and its gender bias (specifically, occupational gender bias). We measure bias in three masked language model families (RoBERTa, DeBERTa, and T5) in two setups: directly using prompt based method, and using a downstream task (Winogender). We find on the one hand that larger models receive higher bias scores on the former task, but when evaluated on the latter, they make fewer gender errors. To examine these potentially conflicting results, we carefully investigate the behavior of the different models on Winogender. We find that while larger models outperform smaller ones, the probability that their mistakes are caused by gender bias is higher. Moreover, we find that the proportion of stereotypical errors compared to anti-stereotypical ones grows with the model size. Our findings highlight the potential risks that can arise from increasing model size.

这项研究检查预先训练模型的大小与其性别偏见（特别是职业性别偏见）之间的关系，发现随着模型大小的增加，性别偏见的得分也增加。尽管较大的模型在 Winogender 任务上性能更好，但其错误可能更容易受到性别偏见的影响，随着模型大小的增加，刻板印象性错误与反对刻板印象性错误的比例也随之增加。

模型规模对性别偏见的影响：错误减少，刻板印象增多？