Recent advancements in Large Language Models(LLMs) have been notable, yet widespread enterprise adoption remains limited due to various constraints. This paper examines bias in LLMs-a crucial issue affecting their usability, reliability, and fairness. Researchers are developing strategies to mitigate bias, including debiasing layers, specialized reference datasets like Winogender and Winobias, and reinforcement learning with human feedback (RLHF). These techniques have been integrated into the latest LLMs. Our study evaluates gender bias in occupational scenarios and gender, age, and racial bias in crime scenarios across four leading LLMs released in 2024: Gemini 1.5 Pro, Llama 3 70B, Claude 3 Opus, and GPT-4o. Findings reveal that LLMs often depict female characters more frequently than male ones in various occupations, showing a 37% deviation from US BLS data. In crime scenarios, deviations from US FBI data are 54% for gender, 28% for race, and 17% for age. We observe that efforts to reduce gender and racial bias often lead to outcomes that may over-index one sub-class, potentially exacerbating the issue. These results highlight the limitations of current bias mitigation techniques and underscore the need for more effective approaches.

本研究探讨了大型语言模型（LLMs）中存在的偏见问题，影响其可用性和公平性。通过分析2024年发布的四种领先LLMs在职业和犯罪场景中的性别、种族和年龄偏见，发现这些模型在职业场景中对女性角色的描绘偏差达37%，而在犯罪场景中性别、种族和年龄的偏差分别为54%、28%和17%。研究结果强调了当前偏见缓解技术的局限性，呼吁寻找更有效的方法。

评估大型语言模型中的性别、种族和年龄偏见：职业和犯罪场景的比较分析