Medical Large Language Models (LLMs) such as ClinicalCamel 70B,
Llama3-OpenBioLLM 70B have demonstrated impressive performance on a wide
variety of medical NLP task.However, there still lacks a large language model
(LLM) specifically designed for cancer domain. Moreover, these LLMs typically
have billions of parameters, making them computationally expensive for
healthcare systems.Thus, in this study, we propose CancerLLM, a model with 7
billion parameters and a Mistral-style architecture, pre-trained on 2,676,642
clinical notes and 515,524 pathology reports covering 17 cancer types, followed
by fine-tuning on three cancer-relevant tasks, including cancer phenotypes
extraction, cancer diagnosis generation, and cancer treatment plan generation.
Our evaluation demonstrated that CancerLLM achieves state-of-the-art results
compared to other existing LLMs, with an average F1 score improvement of 8.1\%.
Additionally, CancerLLM outperforms other models on two proposed robustness
testbeds. This illustrates that CancerLLM can be effectively applied to
clinical AI systems, enhancing clinical research and healthcare delivery in the
field of cancer.

该研究提出了 CancerLLM 模型，该模型具有 70 亿个参数和 Mistral 风格的架构，预先训练了 2,676,642 份临床笔记和 515,524 份病理报告，涵盖了 17 种癌症类型，并在三个与癌症相关的任务上进行了微调，包括癌症表型提取、癌症诊断生成和癌症治疗计划生成。评估结果表明，CancerLLM 相对于其他现有的语言模型取得了最先进的结果，平均 F1 得分提高了 8.1％。此外，CancerLLM 在两个提出的鲁棒性测试中表现优于其他模型，这表明 CancerLLM 可以有效应用于临床 AI 系统，增强了癌症领域的临床研究和医疗服务。