Pretrained large language models (LLMs) are currently state-of-the-art for
solving the vast majority of natural language processing tasks. While many
real-world applications still require fine-tuning to reach satisfactory levels
of performance, many of them are in the low-data regime, making fine-tuning
challenging. To address this, we propose LLM2LLM, a targeted and iterative data
augmentation strategy that uses a teacher LLM to enhance a small seed dataset
by augmenting additional data that can be used for fine-tuning on a specific
task. LLM2LLM (1) fine-tunes a baseline student LLM on the initial seed data,
(2) evaluates and extracts data points that the model gets wrong, and (3) uses
a teacher LLM to generate synthetic data based on these incorrect data points,
which are then added back into the training data. This approach amplifies the
signal from incorrectly predicted data points by the LLM during training and
reintegrates them into the dataset to focus on more challenging examples for
the LLM. Our results show that LLM2LLM significantly enhances the performance
of LLMs in the low-data regime, outperforming both traditional fine-tuning and
other data augmentation baselines. LLM2LLM reduces the dependence on
labor-intensive data curation and paves the way for more scalable and
performant LLM solutions, allowing us to tackle data-constrained domains and
tasks. We achieve improvements up to 24.2% on the GSM8K dataset, 32.6% on
CaseHOLD, 32.0% on SNIPS, 52.6% on TREC and 39.8% on SST-2 over regular
fine-tuning in the low-data regime using a LLaMA2-7B student model.

使用以预训练大型语言模型（LLM）为基础的 LLM2LLM 方法，通过数据增强和迭代，显著提高 LLM 在低数据情况下的性能，优于传统的微调和其他数据增强方法，减少了对数据策划的依赖，为更可扩展和高性能的 LLM 解决方案铺平了道路。