There is a significant gap between patient needs and available mental health support today. In this paper, we aim to thoroughly examine the potential of using Large Language Models (LLMs) to assist professional psychotherapy. To this end, we propose a new benchmark, CBT-BENCH, for the systematic evaluation of cognitive behavioral therapy (CBT) assistance. We include three levels of tasks in CBT-BENCH: I: Basic CBT knowledge acquisition, with the task of multiple-choice questions; II: Cognitive model understanding, with the tasks of cognitive distortion classification, primary core belief classification, and fine-grained core belief classification; III: Therapeutic response generation, with the task of generating responses to patient speech in CBT therapy sessions. These tasks encompass key aspects of CBT that could potentially be enhanced through AI assistance, while also outlining a hierarchy of capability requirements, ranging from basic knowledge recitation to engaging in real therapeutic conversations. We evaluated representative LLMs on our benchmark. Experimental results indicate that while LLMs perform well in reciting CBT knowledge, they fall short in complex real-world scenarios requiring deep analysis of patients' cognitive structures and generating effective responses, suggesting potential future work.

本研究针对目前心理健康支持中患者需求与可用资源之间的显著差距，提出了CBT-BENCH基准，用于系统评估大型语言模型（LLMs）在认知行为治疗（CBT）中的辅助潜力。研究发现，虽然LLMs在基本知识回忆方面表现良好，但在需要深入分析患者认知结构及生成有效反应的复杂真实情境中，它们的表现不足，突显了未来研究的方向。

CBT基准：评估大型语言模型在认知行为治疗中的辅助作用