Pretrained language models have served as important backbones for natural
language processing. Recently, in-domain pretraining has been shown to benefit
various domain-specific downstream tasks. In the biomedical domain, natural
language generation (NLG) tasks are of critical importance, while understudied.
Approaching natural language understanding (NLU) tasks as NLG achieves
satisfying performance in the general domain through constrained language
generation or language prompting. We emphasize the lack of in-domain generative
language models and the unsystematic generative downstream benchmarks in the
biomedical domain, hindering the development of the research community. In this
work, we introduce the generative language model BioBART that adapts BART to
the biomedical domain. We collate various biomedical language generation tasks
including dialogue, summarization, entity linking, and named entity
recognition. BioBART pretrained on PubMed abstracts has enhanced performance
compared to BART and set strong baselines on several tasks. Furthermore, we
conduct ablation studies on the pretraining tasks for BioBART and find that
sentence permutation has negative effects on downstream tasks.

本研究介绍了一种 BioBART 生成语言模型，适应于生物医学领域，用于对话、摘要、实体链接和命名实体识别等各种自然语言生成任务，预先培训在 PubMed 摘要上的 BioBART 比 BART 性能更出色，并在几个任务上设定了强基线。预处理任务的消融研究表明，句子置换对下游任务有负面影响。