Large Language Models (LLMs) have made substantial progress in the past
several months, shattering state-of-the-art benchmarks in many domains. This
paper investigates LLMs' behavior with respect to gender stereotypes, a known
issue for prior models. We use a simple paradigm to test the presence of gender
bias, building on but differing from WinoBias, a commonly used gender bias
dataset, which is likely to be included in the training data of current LLMs.
We test four recently published LLMs and demonstrate that they express biased
assumptions about men and women's occupations. Our contributions in this paper
are as follows: (a) LLMs are 3-6 times more likely to choose an occupation that
stereotypically aligns with a person's gender; (b) these choices align with
people's perceptions better than with the ground truth as reflected in official
job statistics; (c) LLMs in fact amplify the bias beyond what is reflected in
perceptions or the ground truth; (d) LLMs ignore crucial ambiguities in
sentence structure 95% of the time in our study items, but when explicitly
prompted, they recognize the ambiguity; (e) LLMs provide explanations for their
choices that are factually inaccurate and likely obscure the true reason behind
their predictions. That is, they provide rationalizations of their biased
behavior. This highlights a key property of these models: LLMs are trained on
imbalanced datasets; as such, even with the recent successes of reinforcement
learning with human feedback, they tend to reflect those imbalances back at us.
As with other types of societal biases, we suggest that LLMs must be carefully
tested to ensure that they treat minoritized individuals and communities
equitably.

通过对四个最近发表的大型语言模型进行测试，我们发现大型语言模型在性别刻板印象、职业偏见和句子结构等方面表现出偏见，它们更倾向于选择与性别刻板印象相关的职业，并且在提供自己选择的解释时常常存在事实错误，对于这些模型的偏见行为，我们需要谨慎测试以确保它们对待边缘化个体和社区是公平的。