Large language models (LLMs) are increasingly being used in human-centered social scientific tasks, such as data annotation, synthetic data creation, and engaging in dialog. However, these tasks are highly subjective and dependent on human factors, such as one's environment, attitudes, beliefs, and lived experiences. Thus, employing LLMs (which do not have such human factors) in these tasks may result in a lack of variation in data, failing to reflect the diversity of human experiences. In this paper, we examine the role of prompting LLMs with human-like personas and asking the models to answer as if they were a specific human. This is done explicitly, with exact demographics, political beliefs, and lived experiences, or implicitly via names prevalent in specific populations. The LLM personas are then evaluated via (1) subjective annotation task (e.g., detecting toxicity) and (2) a belief generation task, where both tasks are known to vary across human factors. We examine the impact of explicit vs. implicit personas and investigate which human factors LLMs recognize and respond to. Results show that LLM personas show mixed results when reproducing known human biases, but generate generally fail to demonstrate implicit biases. We conclude that LLMs lack the intrinsic cognitive mechanisms of human thought, while capturing the statistical patterns of how people speak, which may restrict their effectiveness in complex social science applications.

通过以人类为模版的角色对大型语言模型进行提示和回答问题，我们研究了这种模型在主观注释任务和信念生成任务中的表现，结果显示大型语言模型在模拟已知的人类偏见方面存在不同的结果，但在表现隐含的偏见方面通常未能达到预期。我们得出结论，大型语言模型缺乏人类思维的内在认知机制，虽然能够捕获人们言语的统计模式，但在复杂社会科学应用中可能限制其效果。

明示和隐示的大型语言模型角色生成意见，但无法复制更深层次的认知和偏见