The conceptualization and categorization of natural objects in the human mind
have long intrigued cognitive scientists and neuroscientists, offering crucial
insights into human perception and cognition. Recently, the rapid development
of Large Language Models (LLMs) has raised the attractive question of whether
these models can also develop human-like object representations through
exposure to vast amounts of linguistic and multimodal data. In this study, we
combined behavioral and neuroimaging analysis methods to uncover how the object
concept representations in LLMs correlate with those of humans. By collecting
large-scale datasets of 4.7 million triplet judgments from LLM and Multimodal
LLM (MLLM), we were able to derive low-dimensional embeddings that capture the
underlying similarity structure of 1,854 natural objects. The resulting
66-dimensional embeddings were found to be highly stable and predictive, and
exhibited semantic clustering akin to human mental representations.
Interestingly, the interpretability of the dimensions underlying these
embeddings suggests that LLM and MLLM have developed human-like conceptual
representations of natural objects. Further analysis demonstrated strong
alignment between the identified model embeddings and neural activity patterns
in many functionally defined brain ROIs (e.g., EBA, PPA, RSC and FFA). This
provides compelling evidence that the object representations in LLMs, while not
identical to those in the human, share fundamental commonalities that reflect
key schemas of human conceptual knowledge. This study advances our
understanding of machine intelligence and informs the development of more
human-like artificial cognitive systems.

通过分析行为和神经成像数据，本研究揭示了大型语言模型（LLMs）中的对象概念表示与人类的相关性，证明了 LLMs 和多模式 LLMs 已经发展出类似于人类的概念性对象表示，该研究推动了对机器智能的理解并为开发更类似人类的人工认知系统提供了参考。