Recently, the increasing demand for superior medical services has highlighted
the discrepancies in the medical infrastructure. With big data, especially
texts, forming the foundation of medical services, there is an exigent need for
effective natural language processing (NLP) solutions tailored to the
healthcare domain. Conventional approaches leveraging pre-trained models
present promising results in this domain and current large language models
(LLMs) offer advanced foundation for medical text processing. However, most
medical LLMs are trained only with supervised fine-tuning (SFT), even though it
efficiently empowers LLMs to understand and respond to medical instructions but
is ineffective in learning domain knowledge and aligning with human preference.
Another engineering barrier that prevents current medical LLM from better text
processing ability is their restricted context length (e.g., 2,048 tokens),
making it hard for the LLMs to process long context, which is frequently
required in the medical domain. In this work, we propose ChiMed-GPT, a new
benchmark LLM designed explicitly for Chinese medical domain, with enlarged
context length to 4,096 tokens and undergoes a comprehensive training regime
with pre-training, SFT, and RLHF. Evaluations on real-world tasks including
information extraction, question answering, and dialogue generation demonstrate
ChiMed-GPT's superior performance over general domain LLMs. Furthermore, we
analyze possible biases through prompting ChiMed-GPT to perform attitude scales
regarding discrimination of patients, so as to contribute to further
responsible development of LLMs in the medical domain. The code and model are
released at this https URL

近年来，对优质医疗服务的需求日益增长，凸显了医疗基础设施中的差异。随着大数据，特别是文本，成为医疗服务的基石，迫切需要专为医疗领域量身定制的有效自然语言处理（NLP）解决方案。我们提出了 ChiMed-GPT，这是一个专为中国医疗领域设计的新型基准大语言模型，其上下文长度扩展为 4,096 个标记，并经历了全面的预训练、有监督微调和强化学习高强度训练过程。在信息提取、问题回答和对话生成等真实世界任务中的评估结果表明，ChiMed-GPT 在医疗领域的性能优于一般领域的大语言模型。此外，通过要求 ChiMed-GPT 执行涉及对患者歧视的态度评定，我们分析了可能存在的偏见，以促进医疗领域大语言模型的负责任开发。代码和模型已经在此 URL 发布。