The rapid advancement of large language models has revolutionized various
applications but also raised crucial concerns about their potential to
perpetuate biases and unfairness when deployed in social media contexts.
Evaluating LLMs' potential biases and fairness has become crucial, as existing
methods rely on limited prompts focusing on just a few groups, lacking a
comprehensive categorical perspective. In this paper, we propose evaluating LLM
biases from a group fairness lens using a novel hierarchical schema
characterizing diverse social groups. Specifically, we construct a dataset,
GFair, encapsulating target-attribute combinations across multiple dimensions.
In addition, we introduce statement organization, a new open-ended text
generation task, to uncover complex biases in LLMs. Extensive evaluations of
popular LLMs reveal inherent safety concerns. To mitigate the biases of LLM
from a group fairness perspective, we pioneer a novel chain-of-thought method
GF-Think to mitigate biases of LLMs from a group fairness perspective.
Experimental results demonstrate its efficacy in mitigating bias in LLMs to
achieve fairness.

通过使用划分群体公平性的全新层次模式，我们提出了对大型语言模型进行偏见评估的方法，并构建了一个跨多个维度的目标属性组合的数据集，GFair。此外，我们还引入了一项新的开放式文本生成任务来揭示大型语言模型中的复杂偏见。通过对流行的大型语言模型进行广泛评估，我们发现存在固有的安全问题。为了从群体公平性的角度减轻大型语言模型的偏见，我们首创了一种新的思维链 (GF-Think) 方法。实验结果表明，此方法在减轻大型语言模型中的偏见以实现公平性方面非常有效。