Lab results are often confusing and hard to understand. Large language models
(LLMs) such as ChatGPT have opened a promising avenue for patients to get their
questions answered. We aim to assess the feasibility of using LLMs to generate
relevant, accurate, helpful, and unharmful responses to lab test-related
questions asked by patients and to identify potential issues that can be
mitigated with augmentation approaches. We first collected lab test results
related question and answer data from Yahoo! Answers and selected 53 QA pairs
for this study. Using the LangChain framework and ChatGPT web portal, we
generated responses to the 53 questions from four LLMs including GPT-4, Meta
LLaMA 2, MedAlpaca, and ORCA_mini. We first assessed the similarity of their
answers using standard QA similarity-based evaluation metrics including ROUGE,
BLEU, METEOR, BERTScore. We also utilized an LLM-based evaluator to judge
whether a target model has higher quality in terms of relevance, correctness,
helpfulness, and safety than the baseline model. Finally, we performed a manual
evaluation with medical experts for all the responses to seven selected
questions on the same four aspects. The results of Win Rate and medical expert
evaluation both showed that GPT-4's responses achieved better scores than all
the other LLM responses and human responses on all four aspects (relevance,
correctness, helpfulness, and safety). However, LLM responses occasionally also
suffer from a lack of interpretation in one's medical context, incorrect
statements, and lack of references. We find that compared to other three LLMs
and human answer from the Q&A website, GPT-4's responses are more accurate,
helpful, relevant, and safer. However, there are cases which GPT-4 responses
are inaccurate and not individualized. We identified a number of ways to
improve the quality of LLM responses.

利用大型语言模型 (LLMs) 为患者提供对实验室检测相关问题的相关、准确、有帮助和安全的回答的可行性进行评估，并通过采用增强方法来解决潜在问题。我们通过从 Yahoo! Answers 收集实验室测试问题和答案数据，并使用 LangChain 框架和 ChatGPT 网络门户为 53 个问题生成了来自 GPT-4、Meta LLaMA 2、MedAlpaca 和 ORCA_mini 四个 LLM 的回答。通过评估相似度和医学专家的人工评估，结果表明 GPT-4 的回答在相关性、准确性、有帮助性和安全性方面优于其他 LLMs 和人工回答，但偶尔也会出现对医学背景的缺乏解释、错误陈述和缺乏参考等问题。我们发现了提高 LLM 回答质量的多种方法。