Large language models (LLMs) are currently at the forefront of intertwining
AI systems with human communication and everyday life. Therefore, it is of
great importance to evaluate their emerging abilities. In this study, we show
that LLMs, most notably GPT-3, exhibit behavior that strikingly resembles
human-like intuition -- and the cognitive errors that come with it. However,
LLMs with higher cognitive capabilities, in particular ChatGPT and GPT-4,
learned to avoid succumbing to these errors and perform in a hyperrational
manner. For our experiments, we probe LLMs with the Cognitive Reflection Test
(CRT) as well as semantic illusions that were originally designed to
investigate intuitive decision-making in humans. Moreover, we probe how sturdy
the inclination for intuitive-like decision-making is. Our study demonstrates
that investigating LLMs with methods from psychology has the potential to
reveal otherwise unknown emergent traits.

本研究通过使用心理学方法，展示了大型语言模型（LLMs），尤其是 GPT-3，表现出类似于人类直觉的行为和认知错误，而具有更高认知能力的 LLMs，特别是 ChatGPT 和 GPT-4，学会避免这些错误，并表现出超理性的方式；此外，我们还探究了直觉决策倾向的稳定程度。