Large language models (LLMs) can pass explicit bias tests but still harbor
implicit biases, similar to humans who endorse egalitarian beliefs yet exhibit
subtle biases. Measuring such implicit biases can be a challenge: as LLMs
become increasingly proprietary, it may not be possible to access their
embeddings and apply existing bias measures; furthermore, implicit biases are
primarily a concern if they affect the actual decisions that these systems
make. We address both of these challenges by introducing two measures of bias
inspired by psychology: LLM Implicit Association Test (IAT) Bias, which is a
prompt-based method for revealing implicit bias; and LLM Decision Bias for
detecting subtle discrimination in decision-making tasks. Using these measures,
we found pervasive human-like stereotype biases in 6 LLMs across 4 social
domains (race, gender, religion, health) and 21 categories (weapons, guilt,
science, career among others). Our prompt-based measure of implicit bias
correlates with embedding-based methods but better predicts downstream
behaviors measured by LLM Decision Bias. This measure is based on asking the
LLM to decide between individuals, motivated by psychological results
indicating that relative not absolute evaluations are more related to implicit
biases. Using prompt-based measures informed by psychology allows us to
effectively expose nuanced biases and subtle discrimination in proprietary LLMs
that do not show explicit bias on standard benchmarks.

通过心理学启发的 LLM 暗示联想测试偏差和 LLM 决策偏差等两项偏差测量方法，揭示了大规模语言模型中普遍存在的人类化刻板印象偏差，以及对决策任务中的微妙歧视进行检测。

在显式无偏的大型语言模型中测量隐性偏见

Measuring Implicit Bias in Explicitly Unbiased Large Language Models

The complete reason behind a decision is a Boolean formula that characterizes
why the decision was made. This recently introduced notion has a number of
applications, which include generating explanations, detecting decision bias
and evaluating counterfactual queries. Prime implicants of the complete reason
are known as sufficient reasons for the decision and they correspond to what is
known as PI explanations and abductive explanations. In this paper, we refer to
the prime implicates of a complete reason as necessary reasons for the
decision. We justify this terminology semantically and show that necessary
reasons correspond to what is known as contrastive explanations. We also study
the computation of complete reasons for multi-class decision trees and graphs
with nominal and numeric features for which we derive efficient, closed-form
complete reasons. We further investigate the computation of shortest necessary
and sufficient reasons for a broad class of complete reasons, which include the
derived closed forms and the complete reasons for Sentential Decision Diagrams
(SDDs). We provide an algorithm which can enumerate their shortest necessary
reasons in output polynomial time. Enumerating shortest sufficient reasons for
this class of complete reasons is hard even for a single reason. For this
problem, we provide an algorithm that appears to be quite efficient as we show
empirically.

本文介绍了决策的完整原因的概念和应用，并研究了其计算方法，包括多类决策树和带有名义和数值特征的图形的完整原因的计算、最短必要和充分原因算法的探讨。

关于必要和充分解释的计算

On the Computation of Necessary and Sufficient Explanations

Recent work has shown that some common machine learning classifiers can be
compiled into Boolean circuits that have the same input-output behavior. We
present a theory for unveiling the reasons behind the decisions made by Boolean
classifiers and study some of its theoretical and practical implications. We
define notions such as sufficient, necessary and complete reasons behind
decisions, in addition to classifier and decision bias. We show how these
notions can be used to evaluate counterfactual statements such as "a decision
will stick even if ... because ... ." We present efficient algorithms for
computing these notions, which are based on new advances on tractable Boolean
circuits, and illustrate them using a case study.

本文探讨机器学习分类器编译成布尔电路同样具有输入 - 输出行为，定义二进制分类器决策背后的决策原因，并阐述其在应用层面的影响。此外，还提出了 “充分的”，“必要的” 和 “完整的” 决策原因的概念，以及对分类器和决策偏差的定义。根据这些概念，我们提出了高效的算法，并使用案例研究来论证它们的实用性。