Human commonsense understanding of the physical and social world is organized
around intuitive theories. These theories support making causal and moral
judgments. When something bad happens, we naturally ask: who did what, and why?
A rich literature in cognitive science has studied people's causal and moral
intuitions. This work has revealed a number of factors that systematically
influence people's judgments, such as the violation of norms and whether the
harm is avoidable or inevitable. We collected a dataset of stories from 24
cognitive science papers and developed a system to annotate each story with the
factors they investigated. Using this dataset, we test whether large language
models (LLMs) make causal and moral judgments about text-based scenarios that
align with those of human participants. On the aggregate level, alignment has
improved with more recent LLMs. However, using statistical analyses, we find
that LLMs weigh the different factors quite differently from human
participants. These results show how curated, challenge datasets combined with
insights from cognitive science can help us go beyond comparisons based merely
on aggregate metrics: we uncover LLMs implicit tendencies and show to what
extent these align with human intuitions.

人类对物理和社会世界的常识性理解建立在直觉理论的基础上，而这些理论支持我们进行因果和道德判断。本文通过收集一系列的故事并对其进行标注，测试了大型语言模型对文本场景进行的因果和道德判断是否与人类参与者的判断相符。结果显示，虽然整体上随着近期大型语言模型的发展，其与人类的判断相符度有所提高，但通过统计分析发现大型语言模型与人类参与者之间对不同因素的重视程度存在明显差异。这些结果显示了如何通过策划和挑战性数据集结合来自认知科学的见解，超越基于总体指标的比较，揭示了大型语言模型的隐含倾向，并展示了这些倾向在多大程度上与人类的直觉相一致。