Realistic practice and tailored feedback are key processes for training peer
counselors with clinical skills. However, existing mechanisms of providing
feedback largely rely on human supervision. Peer counselors often lack
mechanisms to receive detailed feedback from experienced mentors, making it
difficult for them to support the large number of people with mental health
issues who use peer counseling. Our work aims to leverage large language models
to provide contextualized and multi-level feedback to empower peer counselors,
especially novices, at scale. To achieve this, we co-design with a group of
senior psychotherapy supervisors to develop a multi-level feedback taxonomy,
and then construct a publicly available dataset with comprehensive feedback
annotations of 400 emotional support conversations. We further design a
self-improvement method on top of large language models to enhance the
automatic generation of feedback. Via qualitative and quantitative evaluation
with domain experts, we demonstrate that our method minimizes the risk of
potentially harmful and low-quality feedback generation which is desirable in
such high-stakes scenarios.

利用大规模语言模型为初级精神治疗督导员提供多层次背景知识和文本反馈，以此提升大量使用同伴咨询的人群心理健康支持，通过与领域专家的定性和定量评估，确保高风险情境下潜在有害和低质量反馈的最小化。

利用大型语言模型进行多级反馈生成以赋予新手同行辅导员能力

Multi-Level Feedback Generation with Large Language Models for  Empowering Novice Peer Counselors

Millions of users come to online peer counseling platforms to seek support on
diverse topics ranging from relationship stress to anxiety. However, studies
show that online peer support groups are not always as effective as expected
largely due to users' negative experiences with unhelpful counselors. Peer
counselors are key to the success of online peer counseling platforms, but most
of them often do not have systematic ways to receive guidelines or supervision.
In this work, we introduce CARE: an interactive AI-based tool to empower peer
counselors through automatic suggestion generation. During the practical
training stage, CARE helps diagnose which specific counseling strategies are
most suitable in the given context and provides tailored example responses as
suggestions. Counselors can choose to select, modify, or ignore any suggestion
before replying to the support seeker. Building upon the Motivational
Interviewing framework, CARE utilizes large-scale counseling conversation data
together with advanced natural language generation techniques to achieve these
functionalities. We demonstrate the efficacy of CARE by performing both
quantitative evaluations and qualitative user studies through simulated chats
and semi-structured interviews. We also find that CARE especially helps novice
counselors respond better in challenging situations.

本研究介绍了 CARE：一种交互式的基于人工智能生成建议的工具，利用 Motivational Interviewing 框架和大规模的对话数据，帮助在线同侪咨询平台的同伴辅导员诊断哪些具体的辅导策略在特定情况下最适合，并提供个性化的参考响应，从而改善同伴咨询的质量。