Large language models (LLM) have been successful in several natural language
understanding tasks and could be relevant for natural language processing
(NLP)-based mental health application research. In this work, we report the
performance of LLM-based ChatGPT (with gpt-3.5-turbo backend) in three
text-based mental health classification tasks: stress detection (2-class
classification), depression detection (2-class classification), and suicidality
detection (5-class classification). We obtained annotated social media posts
for the three classification tasks from public datasets. Then ChatGPT API
classified the social media posts with an input prompt for classification. We
obtained F1 scores of 0.73, 0.86, and 0.37 for stress detection, depression
detection, and suicidality detection, respectively. A baseline model that
always predicted the dominant class resulted in F1 scores of 0.35, 0.60, and
0.19. The zero-shot classification accuracy obtained with ChatGPT indicates a
potential use of language models for mental health classification tasks.

该研究评估了基于大型语言模型 ChatGPT 在三种文本分类任务中的表现，并发现零 - shot 分类准确度表明语言模型对心理健康分类任务具有潜在的应用价值。