Large language models (LLM) have achieved impressive performance on medical
question-answering benchmarks. However, high benchmark accuracy does not imply
that the performance generalizes to real-world clinical settings. Medical
question-answering benchmarks rely on assumptions consistent with quantifying
LLM performance but that may not hold in the open world of the clinic. Yet LLMs
learn broad knowledge that can help the LLM generalize to practical conditions
regardless of unrealistic assumptions in celebrated benchmarks. We seek to
quantify how well LLM medical question-answering benchmark performance
generalizes when benchmark assumptions are violated. Specifically, we present
an adversarial method that we call MedFuzz (for medical fuzzing). MedFuzz
attempts to modify benchmark questions in ways aimed at confounding the LLM. We
demonstrate the approach by targeting strong assumptions about patient
characteristics presented in the MedQA benchmark. Successful "attacks" modify a
benchmark item in ways that would be unlikely to fool a medical expert but
nonetheless "trick" the LLM into changing from a correct to an incorrect
answer. Further, we present a permutation test technique that can ensure a
successful attack is statistically significant. We show how to use performance
on a "MedFuzzed" benchmark, as well as individual successful attacks. The
methods show promise at providing insights into the ability of an LLM to
operate robustly in more realistic settings.

使用对抗性方法 MedFuzz 对医学问题回答基准中的模型进行干扰，通过修改问题来迷惑大型语言模型 (Large Language Models, LLM)，检验其在违反基准假设时的性能泛化情况，并通过置换检验技术确保成功攻击的统计显著性。这些方法在更加真实的环境中为 LLM 的稳健运行提供了有希望的见解。