Large language models (LLMs) are becoming increasingly relevant as a
potential tool for healthcare, aiding communication between clinicians,
researchers, and patients. However, traditional evaluations of LLMs on medical
exam questions do not reflect the complexity of real patient-doctor
interactions. An example of this complexity is the introduction of patient
self-diagnosis, where a patient attempts to diagnose their own medical
conditions from various sources. While the patient sometimes arrives at an
accurate conclusion, they more often are led toward misdiagnosis due to the
patient's over-emphasis on bias validating information. In this work we present
a variety of LLMs with multiple-choice questions from United States medical
board exams which are modified to include self-diagnostic reports from
patients. Our findings highlight that when a patient proposes incorrect
bias-validating information, the diagnostic accuracy of LLMs drop dramatically,
revealing a high susceptibility to errors in self-diagnosis.

大语言模型在医疗保健领域逐渐变得越来越重要，它可作为一种潜在工具，帮助临床医生、研究人员和患者之间进行沟通。然而，对医学考试问题上的 LLMs 进行传统评估并不能反映真实患者和医生之间互动的复杂性，其中引入了患者自我诊断的情况。本研究通过修改美国医学考试中的多项选择题，将患者的自诊断报告纳入其中，提出了多种 LLMs。研究结果表明，当患者提出错误的偏见验证信息时，LLMs 的诊断准确性显著下降，自诊断存在较高的错误敏感性。