This study examines the effect of prompt engineering on the performance of
Large Language Models (LLMs) in clinical note generation. We introduce an
Automatic Prompt Optimization (APO) framework to refine initial prompts and
compare the outputs of medical experts, non-medical experts, and APO-enhanced
GPT3.5 and GPT4. Results highlight GPT4 APO's superior performance in
standardizing prompt quality across clinical note sections. A human-in-the-loop
approach shows that experts maintain content quality post-APO, with a
preference for their own modifications, suggesting the value of expert
customization. We recommend a two-phase optimization process, leveraging
APO-GPT4 for consistency and expert input for personalization.

研究使用提示工程来提高临床笔记生成中大型语言模型的性能，引入一种自动提示优化（APO）框架来改进初始提示，并比较医学专家、非医学专家以及 APO 增强的 GPT3.5 和 GPT4 的输出结果。结果突显了 GPT4 APO 在标准化临床笔记各个部分的提示质量方面表现优越。人机协同方法表明专家在 APO 后保持内容质量，偏好对自己的修改，这表明专家定制化的价值。我们建议采用两阶段的优化过程，利用 APO-GPT4 提高一致性，并利用专家意见进行个性化。

医生是否了解如何提示？在临床笔记生成中自动优化提示帮助的需求

Do Physicians Know How to Prompt? The Need for Automatic Prompt  Optimization Help in Clinical Note Generation

This paper describes our submission to the MEDIQA-Chat 2023 shared task for
automatic clinical note generation from doctor-patient conversations. We report
results for two approaches: the first fine-tunes a pre-trained language model
(PLM) on the shared task data, and the second uses few-shot in-context learning
(ICL) with a large language model (LLM). Both achieve high performance as
measured by automatic metrics (e.g. ROUGE, BERTScore) and ranked second and
first, respectively, of all submissions to the shared task. Expert human
scrutiny indicates that notes generated via the ICL-based approach with GPT-4
are preferred about as often as human-written notes, making it a promising path
toward automated note generation from doctor-patient conversations.

本文介绍了我们在 MEDIQA-Chat 2023 共享任务中对于从医生 - 患者对话自动生成临床笔记的提交。我们报道了两种方法的结果：第一种微调共享任务数据上的预训练语言模型（PLM），第二种使用大型语言模型（LLM）进行少量样本的上下文学习（ICL）。两种方法在自动指标（如 ROUGE，BERTScore）上均实现了高性能，并分别排名共享任务的二和一位。专业人士的审查表明，通过基于 ICL 的方法和 GPT-4 生成的笔记与人工书写的笔记一样受欢迎，这使其成为从医生 - 患者对话自动生成笔记的有前途的途径。