Large language models, such as GPT-4 and Med-PaLM, have shown impressive
performance on clinical tasks; however, they require access to compute, are
closed-source, and cannot be deployed on device. Mid-size models such as
BioGPT-large, BioMedLM, LLaMA 2, and Mistral 7B avoid these drawbacks, but
their capacity for clinical tasks has been understudied. To help assess their
potential for clinical use and help researchers decide which model they should
use, we compare their performance on two clinical question-answering (QA)
tasks: MedQA and consumer query answering. We find that Mistral 7B is the best
performing model, winning on all benchmarks and outperforming models trained
specifically for the biomedical domain. While Mistral 7B's MedQA score of 63.0%
approaches the original Med-PaLM, and it often can produce plausible responses
to consumer health queries, room for improvement still exists. This study
provides the first head-to-head assessment of open source mid-sized models on
clinical tasks.

比较了大型语言模型和中型开源模型在临床任务上的表现，发现 Mistral 7B 是表现最佳的模型，并且在临床问题回答任务和消费者查询回答任务上优于专门为生物医学领域训练的模型。