We present SciRIFF (Scientific Resource for Instruction-Following and Finetuning), a dataset of 137K instruction-following demonstrations for 54 tasks covering five essential scientific literature understanding capabilities: information extraction, summarization, question answering, claim verification, and classification. SciRIFF demonstrations are notable for their long input contexts, detailed task specifications, and complex structured outputs. While instruction-following resources are available in specific domains such as clinical medicine and chemistry, SciRIFF is the first dataset focused on extracting and synthesizing information from research literature across a wide range of scientific fields. To demonstrate the utility of SciRIFF, we develop a sample-efficient strategy to adapt a general instruction-following model for science by performing additional finetuning on a mix of general-domain and SciRIFF demonstrations. In evaluations on nine held-out scientific tasks, our model -- called SciTulu -- improves over a strong LLM baseline by 28.1% and 6.5% at the 7B and 70B scales respectively, while maintaining general instruction-following performance within 2% of the baseline. We are optimistic that SciRIFF will facilitate the development and evaluation of LLMs to help researchers navigate the ever-growing body of scientific literature. We release our dataset, model checkpoints, and data processing and evaluation code to enable further research.

SciRIFF是一份包含137K个54项任务的指示跟随演示的科学资源，涵盖了信息提取、摘要、问题回答、声明验证和分类等五个必要的科学文献理解能力，其长输入上下文、详细任务规范和复杂的结构化输出使其成为瞩目的资源。通过在一般领域和SciRIFF演示的混合数据上进行进一步微调，我们使用SciRIFF展示了一个在科学领域上适应的样本高效策略，开发出被称为SciTulu的模型，在九个科学任务的评估中，相对于强基准模型，它在7B和70B尺度上分别提高了28.1%和6.5%，同时保持了基线模型与一般指示跟随性能的2%之内。我们希望SciRIFF能促进LMM的发展和评估，帮助研究人员了解不断增长的科学文献资料。我们发布了数据集、模型检查点以及数据处理和评估代码来促进进一步的研究。

SciRIFF：提升科学文献中模型指令遵循能力的资源