Generalist foundation models such as GPT-4 have displayed surprising
capabilities in a wide variety of domains and tasks. Yet, there is a prevalent
assumption that they cannot match specialist capabilities of fine-tuned models.
For example, most explorations to date on medical competency benchmarks have
leveraged domain-specific training, as exemplified by efforts on BioGPT and
Med-PaLM. We build on a prior study of GPT-4's capabilities on medical
challenge benchmarks in the absence of special training. Rather than using
simple prompting to highlight the model's out-of-the-box capabilities, we
perform a systematic exploration of prompt engineering. We find that prompting
innovation can unlock deeper specialist capabilities and show that GPT-4 easily
tops prior leading results for medical benchmarks. The prompting methods we
explore are general purpose, and make no specific use of domain expertise,
removing the need for expert-curated content. Our experimental design carefully
controls for overfitting during the prompt engineering process. We introduce
Medprompt, based on a composition of several prompting strategies. With
Medprompt, GPT-4 achieves state-of-the-art results on all nine of the benchmark
datasets in the MultiMedQA suite. The method outperforms leading specialist
models such as Med-PaLM 2 by a significant margin with an order of magnitude
fewer calls to the model. Steering GPT-4 with Medprompt achieves a 27%
reduction in error rate on the MedQA dataset over the best methods to date
achieved with specialist models and surpasses a score of 90% for the first
time. Beyond medical problems, we show the power of Medprompt to generalize to
other domains and provide evidence for the broad applicability of the approach
via studies of the strategy on exams in electrical engineering, machine
learning, philosophy, accounting, law, nursing, and clinical psychology.

通过对 GPT-4 的提示工程进行系统性探索，我们发现启示创新可以释放更深层次的专家能力，并且证明了 GPT-4 在医学基准测试中轻松超越了先前的主要结果。使用 Medprompt，GPT-4 在 MultiMedQA 套件的九个基准数据集上取得了最先进的结果。

通用基础模型能否超越专用调优？以医学为例的研究案例

Can Generalist Foundation Models Outcompete Special-Purpose Tuning? Case  Study in Medicine

Cross-modal medical image translation is an essential task for synthesizing
missing modality data for clinical diagnosis. However, current learning-based
techniques have limitations in capturing cross-modal and global features,
restricting their suitability to specific pairs of modalities. This lack of
versatility undermines their practical usefulness, particularly considering
that the missing modality may vary for different cases. In this study, we
present MedPrompt, a multi-task framework that efficiently translates different
modalities. Specifically, we propose the Self-adaptive Prompt Block, which
dynamically guides the translation network towards distinct modalities. Within
this framework, we introduce the Prompt Extraction Block and the Prompt Fusion
Block to efficiently encode the cross-modal prompt. To enhance the extraction
of global features across diverse modalities, we incorporate the Transformer
model. Extensive experimental results involving five datasets and four pairs of
modalities demonstrate that our proposed model achieves state-of-the-art visual
quality and exhibits excellent generalization capability.

跨模态医学图像翻译是合成临床诊断中缺失模态数据的必要任务。本研究提出了 MedPrompt，一种多任务框架，可以高效地翻译不同模态。通过引入自适应提示块、提示提取块和提示融合块，我们能够有效地编码跨模态提示。通过集成 Transformer 模型，增强了对不同模态间全局特征的提取，实验结果表明我们提出的模型在视觉质量和泛化能力方面达到了最先进水平。