The advances of Large Language Models (LLMs) are expanding their utility in
both academic research and practical applications. Recent social science
research has explored the use of these "black-box" LLM agents for simulating
complex social systems and potentially substituting human subjects in
experiments. Our study delves into this emerging domain, investigating the
extent to which LLMs exhibit key social interaction principles, such as social
learning, social preference, and cooperative behavior, in their interactions
with humans and other agents. We develop a novel framework for our study,
wherein classical laboratory experiments involving human subjects are adapted
to use LLM agents. This approach involves step-by-step reasoning that mirrors
human cognitive processes and zero-shot learning to assess the innate
preferences of LLMs. Our analysis of LLM agents' behavior includes both the
primary effects and an in-depth examination of the underlying mechanisms.
Focusing on GPT-4, the state-of-the-art LLM, our analyses suggest that LLM
agents appear to exhibit a range of human-like social behaviors such as
distributional and reciprocity preferences, responsiveness to group identity
cues, engagement in indirect reciprocity, and social learning capabilities.
However, our analysis also reveals notable differences: LLMs demonstrate a
pronounced fairness preference, weaker positive reciprocity, and a more
calculating approach in social learning compared to humans. These insights
indicate that while LLMs hold great promise for applications in social science
research, such as in laboratory experiments and agent-based modeling, the
subtle behavioral differences between LLM agents and humans warrant further
investigation. Careful examination and development of protocols in evaluating
the social behaviors of LLMs are necessary before directly applying these
models to emulate human behavior.

大型语言模型在社交科学研究和实际应用中的作用被不断拓展，然而在与人类和其他代理进行交互时，这些模型展现了一系列人类类似的社交行为，同时也存在一些行为差异，因此需要进一步研究和发展评估协议，以直接应用这些模型来模拟人类行为。