In the expanding field of language model applications, medical knowledge
representation remains a significant challenge due to the specialized nature of
the domain. Large language models, such as GPT-4, obtain reasonable scores on
medical question answering tasks, but smaller models are far behind. In this
work, we introduce a method to improve the proficiency of a small language
model in the medical domain by employing a two-fold approach. We first
fine-tune the model on a corpus of medical textbooks. Then, we use GPT-4 to
generate questions similar to the downstream task, prompted with textbook
knowledge, and use them to fine-tune the model. Additionally, we introduce
ECN-QA, a novel medical question answering dataset containing ``progressive
questions'' composed of related sequential questions. We show the benefits of
our training strategy on this dataset. The study's findings highlight the
potential of small language models in the medical domain when appropriately
fine-tuned. The code and weights are available at
this https URL

通过在医学学科上对小型语言模型进行微调和使用 GPT-4 生成类似下游任务的问题进行微调，我们提出了一种方法来提高医学领域中小型语言模型的熟练度。此外，我们还引入了包含相关问题序列的 “渐进问题” 的 ECN-QA，以展示我们的训练策略的好处。