Training and evaluating language models increasingly requires the
construction of meta-datasets --diverse collections of curated data with clear
provenance. Natural language prompting has recently lead to improved zero-shot
generalization by transforming existing, supervised datasets into a diversity
of novel pretraining tasks, highlighting the benefits of meta-dataset curation.
While successful in general-domain text, translating these data-centric
approaches to biomedical language modeling remains challenging, as labeled
biomedical datasets are significantly underrepresented in popular data hubs. To
address this challenge, we introduce BigBIO a community library of 126+
biomedical NLP datasets, currently covering 12 task categories and 10+
languages. BigBIO facilitates reproducible meta-dataset curation via
programmatic access to datasets and their metadata, and is compatible with
current platforms for prompt engineering and end-to-end few/zero shot language
model evaluation. We discuss our process for task schema harmonization, data
auditing, contribution guidelines, and outline two illustrative use cases:
zero-shot evaluation of biomedical prompts and large-scale, multi-task
learning. BigBIO is an ongoing community effort and is available at
this https URL

我们介绍了 BigBIO 项目，这是一个包含 126 个以上生物医学 NLP 数据集的社区库，可用于训练和评估语言模型，其数据集的元数据和编程访问支持元数据集合的构建，并且支持 End-to-End 的零样本任务。