ChatGPT is a large language model developed by OpenAI. Despite its impressive performance across various tasks, no prior work has investigated its capability in the biomedical domain yet. To this end, this paper aims to evaluate the performance of ChatGPT on various benchmark biomedical tasks, such as relation extraction, document classification, question answering, and summarization. To the best of our knowledge, this is the first work that conducts an extensive evaluation of ChatGPT in the biomedical domain. Interestingly, we find based on our evaluation that in biomedical datasets that have smaller training sets, zero-shot ChatGPT even outperforms the state-of-the-art fine-tuned generative transformer models, such as BioGPT and BioBART. This suggests that ChatGPT's pre-training on large text corpora makes it quite specialized even in the biomedical domain. Our findings demonstrate that ChatGPT has the potential to be a valuable tool for various tasks in the biomedical domain that lack large annotated data.

本文旨在评估ChatGPT在生物医学领域中各种基准任务（如关系提取、文档分类、问答和摘要）的性能，在拥有较小训练集的生物医学数据集中，zero-shot ChatGPT甚至优于最先进的经调优生成变换器模型（如BioGPT和BioBART），这表明ChatGPT在生物医学领域也非常专业，具备成为缺乏大型注释数据的各种生物医学任务的有价值工具的潜力。

ChatGPT在生物医学任务中的评估：与微调生成变换器的零样本比较