In this paper, we introduce a novel psychological benchmark, CPsyExam,
constructed from questions sourced from Chinese language examinations. CPsyExam
is designed to prioritize psychological knowledge and case analysis separately,
recognizing the significance of applying psychological knowledge to real-world
scenarios. From the pool of 22k questions, we utilize 4k to create the
benchmark that offers balanced coverage of subjects and incorporates a diverse
range of case analysis techniques.Furthermore, we evaluate a range of existing
large language models~(LLMs), spanning from open-sourced to API-based models.
Our experiments and analysis demonstrate that CPsyExam serves as an effective
benchmark for enhancing the understanding of psychology within LLMs and enables
the comparison of LLMs across various granularities.

我们介绍了一个新颖的心理学基准测试 CPsyExam，该测试由来自中文考试的问题构成。CPsyExam 着重于将心理学知识和案例分析分开，认识到将心理学知识应用于实际场景的重要性。从 22k 个问题中选取 4k 个问题构成的基准测试，它涵盖了各种学科，结合了多样的案例分析技术。此外，我们评估了一系列现有的大型语言模型（LLMs），从开源到 API-based 模型不一而足。我们的实验和分析表明，CPsyExam 是提高 LLMs 对心理学理解的有效基准测试，同时可以比较各种粒度下的 LLMs。