Medical Large Language Models (LLMs) such as ClinicalCamel 70B, Llama3-OpenBioLLM 70B have demonstrated impressive performance on a wide variety of medical NLP task.However, there still lacks a large language model (LLM) specifically designed for cancer domain. Moreover, these LLMs typically have billions of parameters, making them computationally expensive for healthcare systems.Thus, in this study, we propose CancerLLM, a model with 7 billion parameters and a Mistral-style architecture, pre-trained on 2,676,642 clinical notes and 515,524 pathology reports covering 17 cancer types, followed by fine-tuning on three cancer-relevant tasks, including cancer phenotypes extraction, cancer diagnosis generation, and cancer treatment plan generation. Our evaluation demonstrated that CancerLLM achieves state-of-the-art results compared to other existing LLMs, with an average F1 score improvement of 8.1\%. Additionally, CancerLLM outperforms other models on two proposed robustness testbeds. This illustrates that CancerLLM can be effectively applied to clinical AI systems, enhancing clinical research and healthcare delivery in the field of cancer.

该研究提出了CancerLLM模型，该模型具有70亿个参数和Mistral风格的架构，预先训练了2,676,642份临床笔记和515,524份病理报告，涵盖了17种癌症类型，并在三个与癌症相关的任务上进行了微调，包括癌症表型提取、癌症诊断生成和癌症治疗计划生成。评估结果表明，CancerLLM相对于其他现有的语言模型取得了最先进的结果，平均F1得分提高了8.1％。此外，CancerLLM在两个提出的鲁棒性测试中表现优于其他模型，这表明CancerLLM可以有效应用于临床AI系统，增强了癌症领域的临床研究和医疗服务。

癌症LLM：一个癌症领域的大型语言模型