Instruction tuning is critical to large language models (LLMs) for achieving
better instruction following and task adaptation capabilities but its success
heavily relies on the training data quality. Many recent methods focus on
improving the data quality but often overlook the compatibility of the data
with the student model being finetuned. This paper introduces Selective
Reflection-Tuning, a novel paradigm that synergizes a teacher LLM's reflection
and introspection for improving existing data quality with the data selection
capability of the student LLM, to automatically refine existing
instruction-tuning data. This teacher-student collaboration produces
high-quality and student-compatible instruction-response pairs, resulting in
sample-efficient instruction tuning and LLMs of superior performance. Selective
Reflection-Tuning is a data augmentation and synthesis that generally improves
LLM finetuning and self-improvement without collecting brand-new data. We apply
our method to Alpaca and WizardLM data and achieve much stronger and top-tier
7B and 13B LLMs. Our codes, models, and data will be released at
this https URL

选择性反射微调是一种新型范式，通过结合教师型巨型语言模型的反思和内省以及学生型巨型语言模型的数据选择能力，自动优化现有的指示微调数据，从而生成高质量且与学生模型兼容的指示 - 回答对，提升巨型语言模型微调和自我改进的效率，实现卓越性能的巨型语言模型。