We release and introduce SoftTiger, a clinical large language model (CLaM)
designed as a foundation model for healthcare workflows. The narrative and
unstructured nature of clinical notes is a major obstacle for healthcare
intelligentization. We address a critical problem of structuring clinical notes
into clinical data, according to international interoperability standards. We
collect and annotate data for three critical subtasks, namely, international
patient summary, clinical impression and medical encounter. We then supervised
fine-tuned a state-of-the-art LLM using public and credentialed clinical data.
The training is orchestrated in a way that the target model can first support
basic clinical tasks such as abbreviation expansion and temporal information
extraction, and then learn to perform more complex downstream clinical tasks
such as impression and encounter summary. Moreover, we address, several
modeling challenges in the healthcare context, e.g., extra long context window.
Our blind pairwise evaluation shows that SoftTiger outperforms other popular
open-source models and GPT-3.5, comparable to Gemini-pro, and only has a mild
gap from GPT-4. We believe that LLMs may become a step-stone towards healthcare
digitalization and democratization. Therefore, we publicly release SoftTiger
models at scales of 13 billion and 70 billion parameters, as well as datasets
and code for our innovative scalable evaluation, hopefully, making a
significant contribution to the healthcare industry.

我们介绍了 SoftTiger，这是一个临床大型语言模型（CLaM），旨在成为医疗保健工作流的基础模型。我们解决了将临床笔记按照国际互操作性标准转化为临床数据的关键问题，并通过公开和认证的临床数据进行了监督微调，使目标模型能够支持基本的临床任务，并学习执行更复杂的下游临床任务。此外，我们还解决了医疗保健背景中的模型挑战，并展示了 SoftTiger 在盲目对比评估中优于其他流行的开源模型和 GPT-3.5，与 Gemini-pro 相媲美，仅与 GPT-4 略有差距。我们相信，大型语言模型可能成为医疗保健数字化和民主化的一个基石，因此，我们公开发布了规模为 130 亿和 700 亿参数的 SoftTiger 模型，以及创新的可扩展评估的数据集和代码，希望对医疗保健行业做出重要贡献。