Psychological measurement is essential for mental health, self-understanding,
and personal development. Traditional methods, such as self-report scales and
psychologist interviews, often face challenges with engagement and
accessibility. While game-based and LLM-based tools have been explored to
improve user interest and automate assessment, they struggle to balance
engagement with generalizability. In this work, we propose PsychoGAT
(Psychological Game AgenTs) to achieve a generic gamification of psychological
assessment. The main insight is that powerful LLMs can function both as adept
psychologists and innovative game designers. By incorporating LLM agents into
designated roles and carefully managing their interactions, PsychoGAT can
transform any standardized scales into personalized and engaging interactive
fiction games. To validate the proposed method, we conduct psychometric
evaluations to assess its effectiveness and employ human evaluators to examine
the generated content across various psychological constructs, including
depression, cognitive distortions, and personality traits. Results demonstrate
that PsychoGAT serves as an effective assessment tool, achieving statistically
significant excellence in psychometric metrics such as reliability, convergent
validity, and discriminant validity. Moreover, human evaluations confirm
PsychoGAT's enhancements in content coherence, interactivity, interest,
immersion, and satisfaction.

通过将强大的 LLM 代理人纳入指定角色并精心管理其互动，PsychoGAT 能够将任何标准化量表转化为个性化、引人入胜的互动小说游戏，并通过心理测量评估证明其有效性，人工评估确认其在内容一致性、互动性、趣味性、沉浸度和满意度方面的改进。

心理学的 LLM 智能体：关于游戏化评估的研究

LLM Agents for Psychology: A Study on Gamified Assessments

The question addressed in this paper is: If we present to a user an AI system
that explains how it works, how do we know whether the explanation works and
the user has achieved a pragmatic understanding of the AI? In other words, how
do we know that an explanainable AI system (XAI) is any good? Our focus is on
the key concepts of measurement. We discuss specific methods for evaluating:
(1) the goodness of explanations, (2) whether users are satisfied by
explanations, (3) how well users understand the AI systems, (4) how curiosity
motivates the search for explanations, (5) whether the user's trust and
reliance on the AI are appropriate, and finally, (6) how the human-XAI work
system performs. The recommendations we present derive from our integration of
extensive research literatures and our own psychometric evaluations.

本文研究解释型人工智能（XAI）的性能评估，包括解释质量、用户满意度、用户对人工智能系统的理解度、好奇心的激发、信任及依赖程度以及人机协同的表现。根据广泛的研究文献和心理测量评估结果，提出相应的方法和建议。