The process of instruction tuning aligns pre-trained large language models
(LLMs) with open-domain instructions and human-preferred responses. While
several studies have explored autonomous approaches to distilling and
annotating instructions from more powerful proprietary LLMs, such as ChatGPT,
they often neglect the impact of task distributions and the varying difficulty
of instructions of the training sets. This oversight can lead to imbalanced
knowledge capabilities and poor generalization powers of small student LLMs. To
address this challenge, we introduce Task-Aware Curriculum Planning for
Instruction Refinement (TAPIR), a multi-round distillation framework with
balanced task distributions and dynamic difficulty adjustment. This approach
utilizes an oracle LLM to select instructions that are difficult for a student
LLM to follow and distill instructions with balanced task distributions. By
incorporating curriculum planning, our approach systematically escalates the
difficulty levels, progressively enhancing the student LLM's capabilities. We
rigorously evaluate TAPIR using two widely recognized benchmarks, including
AlpacaEval 2.0 and MT-Bench. The empirical results demonstrate that the student
LLMs, trained with our method and less training data, outperform larger
instruction-tuned models and strong distillation baselines. The improvement is
particularly notable in complex tasks, such as logical reasoning and code
generation.

任务感知课程规划的指导下，使用动态难度调整的多轮精馏框架 TAPIR 提供平衡的任务分布，通过升级难度水平逐步提升学生 LLM 的能力，并经过两个广泛认可的基准测试验证，结果表明使用我们的方法及较少的训练数据训练的学生 LLMs 在复杂任务中（如逻辑推理和代码生成）优于较大的经过指导调整的模型和强大的精馏基准线。