Instruction Fine-Tuning, a method enhancing pre-trained language models'
capabilities from mere next-word prediction to complex instruction following,
often employs a one-off training approach on diverse instruction dataset.
However, this method may not effectively enhance models' adherence to
instructions due to the simultaneous handling of varying instruction
complexities. To address this, we propose a novel phased instruction
fine-tuning (Phased IFT) method, grounded in the hypothesis of progressive
alignment, which posits that the transition of a pre-trained language model
from simple next-word prediction to sophisticated instruction following is a
gradual learning process. Specifically, we obtain the score of difficulty for
each instruction via GPT-4, stratify the instruction data into subsets of
increasing difficulty, and sequentially uptrain on these subsets using the
standard supervised loss. Through extensive experiments on the pre-trained
models Llama-2 7B/13B, and Mistral-7B using the 52K Alpaca instruction data, we
demonstrate that Phased IFT significantly surpasses traditional one-off
instruction fine-tuning (One-off IFT) method in win rate, empirically
validating the progressive alignment hypothesis. Our findings suggest that
Phased IFT offers a simple yet effective pathway for elevating the
instruction-following capabilities of pre-trained language models. Models and
datasets from our experiments are freely available at
this https URL

通过渐进对齐的假设，我们提出了一种新颖的分阶段指令微调（Phased IFT）方法，基于难度评分并使用逐步训练的方式显著地提高了预训练语言模型的指令遵循能力。