Large Language Models (LLMs) have demonstrated exceptional task-solving
capabilities, increasingly adopting roles akin to human-like assistants. The
broader integration of LLMs into society has sparked interest in whether they
manifest psychological attributes, and whether these attributes are
stable-inquiries that could deepen the understanding of their behaviors.
Inspired by psychometrics, this paper presents a framework for investigating
psychology in LLMs, including psychological dimension identification,
assessment dataset curation, and assessment with results validation. Following
this framework, we introduce a comprehensive psychometrics benchmark for LLMs
that covers six psychological dimensions: personality, values, emotion, theory
of mind, motivation, and intelligence. This benchmark includes thirteen
datasets featuring diverse scenarios and item types. Our findings indicate that
LLMs manifest a broad spectrum of psychological attributes. We also uncover
discrepancies between LLMs' self-reported traits and their behaviors in
real-world scenarios. This paper demonstrates a thorough psychometric
assessment of LLMs, providing insights into reliable evaluation and potential
applications in AI and social sciences.

本论文提出了一个研究大语言模型的心理学的框架，并通过心理测试验证，发现大语言模型表现出广泛的心理属性，并揭示了自我报告特征与现实场景中行为之间的差异。这些研究结果对于可靠的评估和人工智能以及社会科学的潜在应用具有重要的见解。

量化 AI 心理学：大型语言模型的心理测量基准

Quantifying AI Psychology: A Psychometrics Benchmark for Large Language  Models

The complex information processing system of humans generates a lot of
objective and subjective evaluations, making the exploration of human cognitive
products of great cutting-edge theoretical value. In recent years, deep
learning technologies, which are inspired by biological brain mechanisms, have
made significant strides in the application of psychological or cognitive
scientific research, particularly in the memorization and recognition of facial
data. This paper investigates through experimental research how neural networks
process and store facial expression data and associate these data with a range
of psychological attributes produced by humans. Researchers utilized deep
learning model VGG16, demonstrating that neural networks can learn and
reproduce key features of facial data, thereby storing image memories.
Moreover, the experimental results reveal the potential of deep learning models
in understanding human emotions and cognitive processes and establish a
manifold visualization interpretation of cognitive products or psychological
attributes from a non-Euclidean space perspective, offering new insights into
enhancing the explainability of AI. This study not only advances the
application of AI technology in the field of psychology but also provides a new
psychological theoretical understanding the information processing of the AI.
The code is available in here: this https URL

通过实验研究探究神经网络如何处理和储存面部表情数据，并将这些数据与人类产生的各种心理属性相联系，表明深度学习模型在理解人类情绪和认知过程方面具有潜力，非欧几里德空间视角下的认知产品或心理属性的多样化可视化解释不仅提供了对 AI 的可解释性的新见解，还推动了 AI 技术在心理学领域的应用和心理信息处理的新的理论理解。