In this paper, we introduce a novel psychological benchmark, CPsyExam,
constructed from questions sourced from Chinese language examinations. CPsyExam
is designed to prioritize psychological knowledge and case analysis separately,
recognizing the significance of applying psychological knowledge to real-world
scenarios. From the pool of 22k questions, we utilize 4k to create the
benchmark that offers balanced coverage of subjects and incorporates a diverse
range of case analysis techniques.Furthermore, we evaluate a range of existing
large language models~(LLMs), spanning from open-sourced to API-based models.
Our experiments and analysis demonstrate that CPsyExam serves as an effective
benchmark for enhancing the understanding of psychology within LLMs and enables
the comparison of LLMs across various granularities.

我们介绍了一个新颖的心理学基准测试 CPsyExam，该测试由来自中文考试的问题构成。CPsyExam 着重于将心理学知识和案例分析分开，认识到将心理学知识应用于实际场景的重要性。从 22k 个问题中选取 4k 个问题构成的基准测试，它涵盖了各种学科，结合了多样的案例分析技术。此外，我们评估了一系列现有的大型语言模型（LLMs），从开源到 API-based 模型不一而足。我们的实验和分析表明，CPsyExam 是提高 LLMs 对心理学理解的有效基准测试，同时可以比较各种粒度下的 LLMs。

CPsyExam：用考试评价心理学的中文基准

CPsyExam: A Chinese Benchmark for Evaluating Psychology using  Examinations

This paper introduces INACIA (Instru\c{c}\~ao Assistida com Intelig\^encia
Artificial), a groundbreaking system designed to integrate Large Language
Models (LLMs) into the operational framework of Brazilian Federal Court of
Accounts (TCU). The system automates various stages of case analysis, including
basic information extraction, admissibility examination, Periculum in mora and
Fumus boni iuris analyses, and recommendations generation. Through a series of
experiments, we demonstrate INACIA's potential in extracting relevant
information from case documents, evaluating its legal plausibility, and
generating judicial recommendations. Utilizing a validation dataset alongside
LLMs, our evaluation methodology presents an innovative approach to assessing
system performance, correlating highly with human judgment. The results
highlight INACIA's proficiency in handling complex legal tasks, indicating its
suitability for augmenting efficiency and judicial fairness within legal
systems. The paper also discusses potential enhancements and future
applications, positioning INACIA as a model for worldwide AI integration in
legal domains.

INACIA 是一个整合大型语言模型（LLMs）到巴西联邦法院（TCU）操作框架中的革命性系统，通过自动化案件分析的各个阶段，提取相关信息、评估其合法性，并生成司法建议。通过一系列实验，论文展示了 INACIA 在处理复杂法律任务方面的潜力，并讨论了其未来的潜在增强和应用。