Unstructured Electronic Health Record (EHR) data often contains critical
information complementary to imaging data that would inform radiologists'
diagnoses. However, time constraints and the large volume of notes frequently
associated with individual patients renders manual perusal of such data to
identify relevant evidence infeasible in practice. Modern Large Language Models
(LLMs) provide a flexible means of interacting with unstructured EHR data, and
may provide a mechanism to efficiently retrieve and summarize unstructured
evidence relevant to a given query. In this work, we propose and evaluate an
LLM (Flan-T5 XXL) for this purpose. Specifically, in a zero-shot setting we
task the LLM to infer whether a patient has or is at risk of a particular
condition; if so, we prompt the model to summarize the supporting evidence.
Enlisting radiologists for manual evaluation, we find that this LLM-based
approach provides outputs consistently preferred to a standard information
retrieval baseline, but we also highlight the key outstanding challenge: LLMs
are prone to hallucinating evidence. However, we provide results indicating
that model confidence in outputs might indicate when LLMs are hallucinating,
potentially providing a means to address this.

使用大型语言模型（Flan-T5 XXL）从非结构化电子健康记录（EHR）数据中检索和总结与特定查询相关的论据，较传统信息检索方法更受放射科医生欢迎，但面临 LLMs 产生虚构证据的挑战。然而，模型的信心可能指示 LLMs 是否存在虚构，为解决此问题提供了潜在方法。