Large Language Models (LLMs) can generate biased and toxic responses. Yet
most prior work on LLM gender bias evaluation requires predefined
gender-related phrases or gender stereotypes, which are challenging to be
comprehensively collected and are limited to explicit bias evaluation. In
addition, we believe that instances devoid of gender-related language or
explicit stereotypes in inputs can still induce gender bias in LLMs. Thus, in
this work, we propose a conditional text generation mechanism without the need
for predefined gender phrases and stereotypes. This approach employs three
types of inputs generated through three distinct strategies to probe LLMs,
aiming to show evidence of explicit and implicit gender biases in LLMs. We also
utilize explicit and implicit evaluation metrics to evaluate gender bias in
LLMs under different strategies. Our experiments demonstrate that an increased
model size does not consistently lead to enhanced fairness and all tested LLMs
exhibit explicit and/or implicit gender bias, even when explicit gender
stereotypes are absent in the inputs.

大型语言模型的性别偏见评估涉及大量预定义的性别相关短语和性别刻板印象，而这些短语和刻板印象的全面收集具有挑战性且限制于显性偏见评估。本文提出了一种无需预定义性别短语和刻板印象的条件文本生成机制，通过三种不同策略生成的三种类型的输入来检测大型语言模型中的显性和隐性性别偏见。我们还使用显性和隐性评估指标来评估不同策略下大型语言模型中的性别偏见。实验证明，模型规模的增加并不一定会提高公平性，所有测试的大型语言模型都表现出显性和 / 或隐性性别偏见，即使输入中没有显性性别刻板印象。