Do large language models (LLMs) have their own worldviews and personality
tendencies? Simulations in which an LLM was asked to answer subjective
questions were conducted more than 1 million times. Comparison of the responses
from different LLMs with real data from the European Social Survey (ESS)
suggests that the effect of prompts on bias and variability is fundamental,
highlighting major cultural, age, and gender biases. Methods for measuring the
difference between LLMs and survey data are discussed, such as calculating
weighted means and a new proposed measure inspired by Jaccard similarity. We
conclude that it is important to analyze the robustness and variability of
prompts before using LLMs to model individual decisions or collective behavior,
as their imitation abilities are approximate at best.

大型语言模型（LLMs）是否具有其自己的世界观和个性倾向？对 LLM 模型回答主观问题的模拟进行了 100 多万次，将其回答与欧洲社会调查（ESS）的真实数据进行比较，结果表明问题引导对偏见和变异的影响是基本的，突出了主要的文化、年龄和性别偏见。文中还讨论了测量 LLMs 与调查数据差异的方法，如计算加权平均值和基于 Jaccard 相似性的一项新提出的测量。我们得出结论，在使用 LLMs 模拟个体决策或集体行为之前，分析问题引导的稳健性和变异性非常重要，因为它们的模拟能力充其量是近似的。

大型语言模型是否具有变色龙特性？

Are Large Language Models Chameleons?

We mathematically compare four competing definitions of group-level
nondiscrimination: demographic parity, equalized odds, predictive parity, and
calibration. Using the theoretical framework of Friedler et al., we study the
properties of each definition under various worldviews, which are assumptions
about how, if at all, the observed data is biased. We argue that different
worldviews call for different definitions of fairness, and we specify the
worldviews that, when combined with the desire to avoid a criterion for
discrimination that we call disparity amplification, motivate demographic
parity and equalized odds. We also argue that predictive parity and calibration
are insufficient for avoiding disparity amplification because predictive parity
allows an arbitrarily large inter-group disparity and calibration is not robust
to post-processing. Finally, we define a worldview that is more realistic than
the previously considered ones, and we introduce a new notion of fairness that
corresponds to this worldview.

通过对四种竞争性的群体非歧视定义：人口平等、等候平等、预测平等、校准；的数学比较，我们研究了不同世界观下，每个定义的属性，指出了不同的世界观需要不同的公平定义，并且界定了一个与更真实的世界观对应的新的公平概念。