Large Language Models (LLMs) have emerged as powerful candidates to inform
clinical decision-making processes. While these models play an increasingly
prominent role in shaping the digital landscape, two growing concerns emerge in
healthcare applications: 1) to what extent do LLMs exhibit social bias based on
patients' protected attributes (like race), and 2) how do design choices (like
architecture design and prompting strategies) influence the observed biases? To
answer these questions rigorously, we evaluated eight popular LLMs across three
question-answering (QA) datasets using clinical vignettes (patient
descriptions) standardized for bias evaluations. We employ red-teaming
strategies to analyze how demographics affect LLM outputs, comparing both
general-purpose and clinically-trained models. Our extensive experiments reveal
various disparities (some significant) across protected groups. We also observe
several counter-intuitive patterns such as larger models not being necessarily
less biased and fined-tuned models on medical data not being necessarily better
than the general-purpose models. Furthermore, our study demonstrates the impact
of prompt design on bias patterns and shows that specific phrasing can
influence bias patterns and reflection-type approaches (like Chain of Thought)
can reduce biased outcomes effectively. Consistent with prior studies, we call
on additional evaluations, scrutiny, and enhancement of LLMs used in clinical
decision support applications.

通过对八种流行的大型语言模型在三个临床病例描述数据集上进行严格评估和分析，我们发现这些模型在受保护群体之间存在各种不同程度的社会偏差，而大小模型并不一定比通用模型更具偏见，经过设计决策的提示方式对社会偏差也有影响，研究呼吁对在临床决策支持应用中使用的大型语言模型进行进一步评估、审查和改进。