The rapid advancement of large language models has revolutionized various applications but also raised crucial concerns about their potential to perpetuate biases and unfairness when deployed in social media contexts. Evaluating LLMs' potential biases and fairness has become crucial, as existing methods rely on limited prompts focusing on just a few groups, lacking a comprehensive categorical perspective. In this paper, we propose evaluating LLM biases from a group fairness lens using a novel hierarchical schema characterizing diverse social groups. Specifically, we construct a dataset, GFair, encapsulating target-attribute combinations across multiple dimensions. In addition, we introduce statement organization, a new open-ended text generation task, to uncover complex biases in LLMs. Extensive evaluations of popular LLMs reveal inherent safety concerns. To mitigate the biases of LLM from a group fairness perspective, we pioneer a novel chain-of-thought method GF-Think to mitigate biases of LLMs from a group fairness perspective. Experimental results demonstrate its efficacy in mitigating bias in LLMs to achieve fairness.

通过使用划分群体公平性的全新层次模式，我们提出了对大型语言模型进行偏见评估的方法，并构建了一个跨多个维度的目标属性组合的数据集，GFair。此外，我们还引入了一项新的开放式文本生成任务来揭示大型语言模型中的复杂偏见。通过对流行的大型语言模型进行广泛评估，我们发现存在固有的安全问题。为了从群体公平性的角度减轻大型语言模型的偏见，我们首创了一种新的思维链(GF-Think)方法。实验结果表明，此方法在减轻大型语言模型中的偏见以实现公平性方面非常有效。

大规模语言模型的团体公平透视