Adapting a language model into a specific domain, a.k.a `domain adaption', is
a common practice when specialized knowledge, e.g. medicine, is not
encapsulated in a general language model like Llama2. The challenge lies in the
heterogeneity of data across the two training stages, as it varies in
languages, genres, or formats. To tackle this and simplify the learning
protocol, we propose to transform heterogeneous data, from the both
pre-training and supervised stages, into a unified, simple input-output pair
format. We validate the new protocol in the domains where proprietary LLMs like
ChatGPT perform relatively poorly, such as Traditional Chinese Medicine. The
developed model, HuatuoGPT-II, has shown state-of-the-art performance in
Chinese medicine domain on a number of benchmarks, e.g. medical licensing
exams. It even outperforms proprietary models like ChatGPT and GPT-4 in some
aspects, especially in Traditional Chinese Medicine. Expert manual evaluations
further validate HuatuoGPT-II's advantages over existing LLMs. Notably,
HuatuoGPT-II was benchmarked in a fresh Chinese National Medical Licensing
Examination where it achieved the best performance, showcasing not only its
effectiveness but also its generalization capabilities.

通过将异构数据转化为统一的简单输入输出对格式，我们提出了一种简化学习协议的方法，用于解决领域适应中的挑战，并验证了这种方法在中医领域的性能优越性，开发的 HuatuoGPT-II 模型在中医领域的表现最优，不仅显示出了其有效性，还展示了其泛化能力。

HuatuoGPT-II, 一阶段训练于医学领域的语言模型适应

HuatuoGPT-II, One-stage Training for Medical Adaption of LLMs

Recent advances in Large Language Models (LLMs) have achieved remarkable
breakthroughs in understanding and responding to user intents. However, their
performance lag behind general use cases in some expertise domains, such as
Chinese medicine. Existing efforts to incorporate Chinese medicine into LLMs
rely on Supervised Fine-Tuning (SFT) with single-turn and distilled dialogue
data. These models lack the ability for doctor-like proactive inquiry and
multi-turn comprehension and cannot always align responses with safety and
professionalism experts. In this work, we introduce Zhongjing, the first
Chinese medical LLaMA-based LLM that implements an entire training pipeline
from pre-training to reinforcement learning with human feedback (RLHF).
Additionally, we introduce a Chinese multi-turn medical dialogue dataset of
70,000 authentic doctor-patient dialogues, CMtMedQA, which significantly
enhances the model's capability for complex dialogue and proactive inquiry
initiation. We define a refined annotation rule and evaluation criteria given
the biomedical domain's unique characteristics. Results show that our model
outperforms baselines in various capacities and matches the performance of
ChatGPT in a few abilities, despite having 50x training data with previous best
model and 100x parameters with ChatGPT. RLHF further improves the model's
instruction-following ability and safety. We also release our code, datasets
and model for further research.

该研究论文介绍了一个基于中医的大型语言模型，通过从事前训练到强化学习与人类反馈（RLHF）的整个训练流程，以及引入一个包含 7 万个真实医患对话的中文多轮医疗对话数据集 CMtMedQA，该模型在多个方面超越了基线模型，并且通过对该模型的进一步改进来提高其指令遵循能力和安全性。