We study GPT-3, a recent large language model, using tools from cognitive
psychology. More specifically, we assess GPT-3's decision-making, information
search, deliberation, and causal reasoning abilities on a battery of canonical
experiments from the literature. We find that much of GPT-3's behavior is
impressive: it solves vignette-based tasks similarly or better than human
subjects, is able to make decent decisions from descriptions, outperforms
humans in a multi-armed bandit task, and shows signatures of model-based
reinforcement learning. Yet we also find that small perturbations to
vignette-based tasks can lead GPT-3 vastly astray, that it shows no signatures
of directed exploration, and that it fails miserably in a causal reasoning
task. These results enrich our understanding of current large language models
and pave the way for future investigations using tools from cognitive
psychology to study increasingly capable and opaque artificial agents.

本文借助认知心理学的工具对最近的大型语言模型 GPT-3 进行了研究评估其在任务决策、信息搜索、思考以及因果推理方面的能力，结果显示 GPT-3 表现出的许多行为令人印象深刻，能够像人类一样解决任务，但也发现了其局限性，如对于因果推理任务失败，这些结果丰富了我们对当前大型语言模型的理解，并为进一步利用认知心理学工具研究越来越能干和难以理解的人工智能代理铺平了道路。