The recent technology boost of large language models (LLMs) has empowered a
variety of applications. However, there is very little research on
understanding and improving LLMs' capability for the mental health domain. In
this work, we present the first comprehensive evaluation of multiple LLMs,
including Alpaca, Alpaca-LoRA, and GPT-3.5, on various mental health prediction
tasks via online text data. We conduct a wide range of experiments, covering
zero-shot prompting, few-shot prompting, and instruction finetuning. The
results indicate the promising yet limited performance of LLMs with zero-shot
and few-shot prompt designs for mental health tasks. More importantly, our
experiments show that instruction finetuning can significantly boost the
performance of LLMs for all tasks simultaneously. Our best-finetuned model,
Mental-Alpaca, outperforms GPT-3.5 (25 times bigger) by 16.7\% on balanced
accuracy and performs on par with the state-of-the-art task-specific model. We
summarize our findings into a set of action guidelines for future researchers,
engineers, and practitioners on how to empower LLMs with better mental health
domain knowledge and become an expert in mental health prediction tasks.

我们在这项工作中，首次对多种大语言模型（LLMs）进行了全面评估，包括 Alpaca、Alpaca-LoRA 和 GPT-3.5，针对在线文本数据中的各种心理健康预测任务。我们进行了广泛的实验，涵盖了零样本提示、少样本提示和指令微调。研究结果表明，对于心理健康任务，LLMs 在零样本和少样本提示设计上具有有限但有希望的性能。更重要的是，我们的实验表明，指令微调可以显著提升 LLMs 在所有任务上的表现。我们最佳微调模型 Mental-Alpaca 在平衡精度上比 GPT-3.5（规模大 25 倍）高出 16.7％，并与最先进的任务特定模型相媲美。我们总结了一系列行动指南，供未来的研究人员、工程师和实践者参考，介绍如何赋予 LLMs 更好的心理健康领域知识，并成为心理健康预测任务的专家。