Evidence suggests that different prompts lead large language models (LLMs) to
generate responses with varying quality. Yet, little is known about prompts'
effects on response quality in healthcare domains. In this exploratory study,
we address this gap, focusing on a specific healthcare domain: dementia
caregiving. We first developed an innovative prompt template with three
components: (1) system prompts (SPs) featuring 4 different roles; (2) an
initialization prompt; and (3) task prompts (TPs) specifying different levels
of details, totaling 12 prompt combinations. Next, we selected 3 social media
posts containing complicated, real-world questions about dementia caregivers'
challenges in 3 areas: memory loss and confusion, aggression, and driving. We
then entered these posts into GPT-4, with our 12 prompts, to generate 12
responses per post, totaling 36 responses. We compared the word count of the 36
responses to explore potential differences in response length. Two experienced
dementia care clinicians on our team assessed the response quality using a
rating scale with 5 quality indicators: factual, interpretation, application,
synthesis, and comprehensiveness (scoring range: 0-5; higher scores indicate
higher quality).

通过使用 12 个提示组合在 GPT-4 中生成 36 个响应来比较 36 个响应的字数，本研究旨在探索不同提示对医疗保健领域中响应质量的影响，关注特定的医疗保健领域：痴呆症护理。