Background: Rapid advancements in natural language processing have led to the
development of large language models with the potential to revolutionize mental
health care. These models have shown promise in assisting clinicians and
providing support to individuals experiencing various psychological challenges.
Objective: This study aims to compare the performance of two large language
models, GPT-4 and Chat-GPT, in responding to a set of 18 psychological prompts,
to assess their potential applicability in mental health care settings.
Methods: A blind methodology was employed, with a clinical psychologist
evaluating the models' responses without knowledge of their origins. The
prompts encompassed a diverse range of mental health topics, including
depression, anxiety, and trauma, to ensure a comprehensive assessment.
Results: The results demonstrated a significant difference in performance
between the two models (p > 0.05). GPT-4 achieved an average rating of 8.29 out
of 10, while Chat-GPT received an average rating of 6.52. The clinical
psychologist's evaluation suggested that GPT-4 was more effective at generating
clinically relevant and empathetic responses, thereby providing better support
and guidance to potential users.
Conclusions: This study contributes to the growing body of literature on the
applicability of large language models in mental health care settings. The
findings underscore the importance of continued research and development in the
field to optimize these models for clinical use. Further investigation is
necessary to understand the specific factors underlying the performance
differences between the two models and to explore their generalizability across
various populations and mental health conditions.

该研究比较了两个大型语言模型 GPT-4 和 Chat-GPT 在回应 18 个心理提示方面的表现，以评估它们在心理健康护理环境中的潜在适用性。结果显示 GPT-4 在生成临床相关和富有同理心的回应方面更有效，提供更好的支持和指导。这项研究为大型语言模型在心理健康护理领域的适用性提供了贡献，强调了在该领域持续研究和开发的重要性。需要进一步研究了解造成两个模型性能差异的具体因素，并探索其在不同人群和心理健康状况中的普适性。

比较 GPT-4 与 Chat-GPT 在心理健康护理中的功效：对大型语言模型进行盲评估以支持心理辅导

Comparing the Efficacy of GPT-4 and Chat-GPT in Mental Health Care: A  Blind Assessment of Large Language Models for Psychological Support

Large language models (LLMs) such as Chat-GPT can produce coherent, cohesive,
relevant, and fluent answers for various natural language processing (NLP)
tasks. Taking document-level machine translation (MT) as a testbed, this paper
provides an in-depth evaluation of LLMs' ability on discourse modeling. The
study fo-cuses on three aspects: 1) Effects of Discourse-Aware Prompts, where
we investigate the impact of different prompts on document-level translation
quality and discourse phenomena; 2) Comparison of Translation Models, where we
compare the translation performance of Chat-GPT with commercial MT systems and
advanced document-level MT methods; 3) Analysis of Discourse Modelling
Abilities, where we further probe discourse knowledge encoded in LLMs and
examine the impact of training techniques on discourse modeling. By evaluating
a number of benchmarks, we surprisingly find that 1) leveraging their powerful
long-text mod-eling capabilities, ChatGPT outperforms commercial MT systems in
terms of human evaluation. 2) GPT-4 demonstrates a strong ability to explain
discourse knowledge, even through it may select incorrect translation
candidates in contrastive testing. 3) ChatGPT and GPT-4 have demonstrated
superior performance and show potential to become a new and promising paradigm
for document-level translation. This work highlights the challenges and
opportunities of discourse modeling for LLMs, which we hope can inspire the
future design and evaluation of LLMs.

本文研究利用 Chat-GPT 建立的大型语言模型在文档级机器翻译中的应用，通过评估话语建模的能力，比较它与商业翻译系统和高级文档级机器翻译方法的性能，发现 Chat-GPT 在人类评估方面表现优异，同时揭示了话语建模的挑战和机遇。