In this study, we investigate the capacity of large language models (LLMs),
specifically GPT-3.5, to operationalise natural language descriptions of
cooperative, competitive, altruistic, and self-interested behavior in social
dilemmas. Our focus is on the iterated Prisoner's Dilemma, a classic example of
a non-zero-sum interaction, but our broader research program encompasses a
range of experimental economics scenarios, including the ultimatum game,
dictator game, and public goods game. Using a within-subject experimental
design, we instantiated LLM-generated agents with various prompts that conveyed
different cooperative and competitive stances. We then assessed the agents'
level of cooperation in the iterated Prisoner's Dilemma, taking into account
their responsiveness to the cooperative or defection actions of their partners.
Our results provide evidence that LLMs can translate natural language
descriptions of altruism and selfishness into appropriate behaviour to some
extent, but exhibit limitations in adapting their behavior based on conditioned
reciprocity. The observed pattern of increased cooperation with defectors and
decreased cooperation with cooperators highlights potential constraints in the
LLM's ability to generalize its knowledge about human behavior in social
dilemmas. We call upon the research community to further explore the factors
contributing to the emergent behavior of LLM-generated agents in a wider array
of social dilemmas, examining the impact of model architecture, training
parameters, and various partner strategies on agent behavior. As more advanced
LLMs like GPT-4 become available, it is crucial to investigate whether they
exhibit similar limitations or are capable of more nuanced cooperative
behaviors, ultimately fostering the development of AI systems that better align
with human values and social norms.

通过实验，本研究发现大型语言模型能够在一定程度上将自然语言描述的利他主义和自私行为转化为适当的行为，但在适应有条件回报的情况下存在局限性，特别是在社会困境的一般人类行为模式方面存在潜在限制。研究呼吁进一步探讨大型语言模型生成的代理在更广泛的社会困境中自动生成行为的因素，探讨模型架构、训练参数和各种合作伙伴策略对代理行为的影响，最终促进更符合人类价值和社会规范的人工智能系统的发展。