Detecting stereotypes and biases in Large Language Models (LLMs) is crucial
for enhancing fairness and reducing adverse impacts on individuals or groups
when these models are applied. Traditional methods, which rely on embedding
spaces or are based on probability metrics, fall short in revealing the nuanced
and implicit biases present in various contexts. To address this challenge, we
propose the FairMonitor framework and adopt a static-dynamic detection method
for a comprehensive evaluation of stereotypes and biases in LLMs. The static
component consists of a direct inquiry test, an implicit association test, and
an unknown situation test, including 10,262 open-ended questions with 9
sensitive factors and 26 educational scenarios. And it is effective for
evaluating both explicit and implicit biases. Moreover, we utilize the
multi-agent system to construst the dynamic scenarios for detecting subtle
biases in more complex and realistic setting. This component detects the biases
based on the interaction behaviors of LLMs across 600 varied educational
scenarios. The experimental results show that the cooperation of static and
dynamic methods can detect more stereotypes and biased in LLMs.

使用 FairMonitor 框架，采用静态 - 动态检测方法对 LLMs 中的刻板印象和偏见进行全面评估。静态组件包括直接问询测试、隐含联想测试和未知情境测试，用于评估显性和隐性偏见。动态组件利用多智能体系统构建动态场景，检测复杂和现实设置中 LLMs 的细微偏见。实验结果表明，静态和动态方法的合作能够检测出更多 LLMs 中的刻板印象和偏见。