Large Language Models (LLMs) such as ChatGPT have received enormous attention
over the past year and are now used by hundreds of millions of people every
day. The rapid adoption of this technology naturally raises questions about the
possible biases such models might exhibit. In this work, we tested one of these
models (GPT-3) on a range of cognitive effects, which are systematic patterns
that are usually found in human cognitive tasks. We found that LLMs are indeed
prone to several human cognitive effects. Specifically, we show that the
priming, distance, SNARC, and size congruity effects were presented with GPT-3,
while the anchoring effect is absent. We describe our methodology, and
specifically the way we converted real-world experiments to text-based
experiments. Finally, we speculate on the possible reasons why GPT-3 exhibits
these effects and discuss whether they are imitated or reinvented.

通过对 GPT-3 模型的测试，我们发现大型语言模型（LLMs）可能会出现几种人类认知效应，包括提前启动效应、距离效应、SNARC 效应和尺寸一致性效应，但缺乏锚定效应。我们描述了我们的方法学，并讨论了 GPT-3 出现这些效应的可能原因以及它们是模拟还是重新创造的问题。

大型语言模型中的认知效应

Cognitive Effects in Large Language Models

Abstract reasoning is a key ability for an intelligent system. Large language
models achieve above-chance performance on abstract reasoning tasks, but
exhibit many imperfections. However, human abstract reasoning is also
imperfect, and depends on our knowledge and beliefs about the content of the
reasoning problem. For example, humans reason much more reliably about logical
rules that are grounded in everyday situations than arbitrary rules about
abstract attributes. The training experiences of language models similarly
endow them with prior expectations that reflect human knowledge and beliefs. We
therefore hypothesized that language models would show human-like content
effects on abstract reasoning problems. We explored this hypothesis across
three logical reasoning tasks: natural language inference, judging the logical
validity of syllogisms, and the Wason selection task (Wason, 1968). We find
that state of the art large language models (with 7 or 70 billion parameters;
Hoffman et al., 2022) reflect many of the same patterns observed in humans
across these tasks -- like humans, models reason more effectively about
believable situations than unrealistic or abstract ones. Our findings have
implications for understanding both these cognitive effects, and the factors
that contribute to language model performance.

大型语言模型在抽象推理任务上表现优秀，但与人类推理问题存在相似的缺陷，同时受到人类知识和信仰的影响，使用可信的现实情境方案进行推理可以提高推理质量，这种关联性帮助我们了解认知效应和语言模型表现的因素。