Large language models exhibit promising general capabilities but often lack
specialized knowledge for domain-specific tasks. Developing domain experts from
a base model enables a range of applications without prohibitive training
costs. This work demonstrates a method using continuous training and
instruction fine-tuning to rapidly adapt Llama 2 base models to the Chinese
medical domain. We first conduct continuous training on 1B tokens from Chinese
medical references to teach relevant vocabulary and knowledge. The models are
then fine-tuned on 54K examples sourced from the Chinese National Medical
Licensing Examination. Experiments on Chinese medical data confirm the
effectiveness of this approach, producing a model comparable to GPT-3.5-turbo
while using way less computational resource. The resulting domain-specific
model could be useful for various Chinese medical applications. More broadly,
this provides a template for domain-specific training of large language models
in areas where pre-trained models lack the required expertise, such as law,
science, and engineering.

使用连续训练和指导微调的方法，快速适应中国医学领域的 Llama 2 基础模型，实验证实了该方法的有效性，产生了与 GPT-3.5-turbo 相媲美的模型，并且使用更少的计算资源。这个领域特定模型对于各种中国医学应用是有用的，并为领域特定训练提供了一个模板，用于那些预训练模型缺乏所需专业知识的领域，如法律、科学和工程。