Large language models (LLMs), trained on vast datasets, can carry biases that
manifest in various forms, from overt discrimination to implicit stereotypes.
One facet of bias is performance disparities in LLMs, often harming
underprivileged groups, such as racial minorities. A common approach to
quantifying bias is to use template-based bias probes, which explicitly state
group membership (e.g. White) and evaluate if the outcome of a task, sentiment
analysis for instance, is invariant to the change of group membership (e.g.
change White race to Black). This approach is widely used in bias
quantification. However, in this work, we find evidence of an unexpectedly
overlooked consequence of using template-based probes for LLM bias
quantification. We find that in doing so, text examples associated with White
ethnicities appear to be classified as exhibiting negative sentiment at
elevated rates. We hypothesize that the scenario arises artificially through a
mismatch between the pre-training text of LLMs and the templates used to
measure bias through reporting bias, unstated norms that imply group membership
without explicit statement. Our finding highlights the potential misleading
impact of varying group membership through explicit mention in bias
quantification

大型语言模型的偏见在量化偏见时可能会导致模板式偏见探测的误导性影响。