In collaboration with Postpartum Support International (PSI), a non-profit
organization dedicated to supporting caregivers with postpartum mood and
anxiety disorders, we developed three chatbots to provide context-specific
empathetic support to postpartum caregivers, leveraging both rule-based and
generative models. We present and evaluate the performance of our chatbots
using both machine-based metrics and human-based questionnaires. Overall, our
rule-based model achieves the best performance, with outputs that are close to
ground truth reference and contain the highest levels of empathy. Human users
prefer the rule-based chatbot over the generative chatbot for its
context-specific and human-like replies. Our generative chatbot also produced
empathetic responses and was described by human users as engaging. However,
limitations in the training dataset often result in confusing or nonsensical
responses. We conclude by discussing practical benefits of rule-based vs.
generative models for supporting individuals with mental health challenges. In
light of the recent surge of ChatGPT and BARD, we also discuss the
possibilities and pitfalls of large language models for digital mental
healthcare.

我们开发了三种聊天机器人，旨在为产后护理者提供特定情境下的共情支持，并通过基于规则和生成模型的指标对其性能进行评估。研究表明，基于规则的模型在共情度和与真实参考接近程度方面表现最佳，而人们更喜欢基于规则的聊天机器人，因为其具备特定情境下的人性回复。然而，我们的生成模型在训练数据集限制下，偶尔会产生混乱或荒谬的回复。最后，我们讨论了支持心理健康挑战个体的基于规则和生成模型的实际好处，并对大型语言模型在数字心理保健领域的可能性和风险进行了讨论。