In response to the pressing need for advanced clinical problem-solving tools
in healthcare, we introduce BooksMed, a novel framework based on a Large
Language Model (LLM). BooksMed uniquely emulates human cognitive processes to
deliver evidence-based and reliable responses, utilizing the GRADE (Grading of
Recommendations, Assessment, Development, and Evaluations) framework to
effectively quantify evidence strength. For clinical decision-making to be
appropriately assessed, an evaluation metric that is clinically aligned and
validated is required. As a solution, we present ExpertMedQA, a multispecialty
clinical benchmark comprised of open-ended, expert-level clinical questions,
and validated by a diverse group of medical professionals. By demanding an
in-depth understanding and critical appraisal of up-to-date clinical
literature, ExpertMedQA rigorously evaluates LLM performance. BooksMed
outperforms existing state-of-the-art models Med-PaLM 2, Almanac, and ChatGPT
in a variety of medical scenarios. Therefore, a framework that mimics human
cognitive stages could be a useful tool for providing reliable and
evidence-based responses to clinical inquiries.

通过模拟人类认知过程，BooksMed 使用大型语言模型（LLM）构建的框架，利用 GRADE 框架来提供基于证据的可靠答复，同时引入 ExpertMedQA 来评估 LLM 的性能，从而成为在临床决策中提供可靠和基于证据的答案的有用工具。