Large Language Models (LLMs) have made significant strides in natural
language processing but face challenges in terms of computational expense and
inefficiency as they grow in size, especially in domain-specific tasks. Small
Language Models (SLMs), on the other hand, often struggle in these tasks due to
limited capacity and training data. In this paper, we introduce Dr. LLaMA, a
method for improving SLMs through generative data augmentation using LLMs,
focusing on medical question-answering tasks and the PubMedQA dataset. Our
findings indicate that LLMs effectively refine and diversify existing
question-answer pairs, resulting in improved performance of a much smaller
model on domain-specific QA datasets after fine-tuning. This study highlights
the challenges of using LLMs for domain-specific question answering and
suggests potential research directions to address these limitations, ultimately
aiming to create more efficient and capable models for specialized
applications. We have also made our code available for interested researchers

介绍了 Dr. LLaMA，这是一种通过使用大型语言模型进行生成式数据增强来改善小型语言模型的方法，主要关注医学问答任务和 PubMedQA 数据集。研究表明，LLMs 可以有效地改善和多样化问题 - 答案对，从而在微调后使得规模更小的模型获得更好的领域特定 QA 数据集性能。该研究特别强调了在领域特定的问答任务中使用 LLMS 所面临的挑战，并建议了解决这些限制的潜在研究方向，旨在创建更高效、更有能力的专门应用模型。