Instruction tuning effectively optimizes Large Language Models (LLMs) for downstream tasks. Due to the changing environment in real-life applications, LLMs necessitate continual task-specific adaptation without catastrophic forgetting. Considering the heavy computational cost, replay-based Continual Learning (CL) methods are the simplest and most widely used for LLMs to address the forgetting issue. However, traditional replay-based methods do not fully utilize instructions to customize the replay strategy. In this work, we propose a novel paradigm called Instruction-based Continual Learning (InsCL). InsCL dynamically replays previous data based on task similarity, calculated by Wasserstein Distance with instructions. Moreover, we further introduce an Instruction Information Metric (InsInfo) to quantify the complexity and diversity of instructions. According to InsInfo, InsCL guides the replay process more inclined to high-quality data. We conduct extensive experiments over 16 tasks with different training orders, observing consistent performance improvements of InsCL. When all tasks have been trained, InsCL achieves performance gains of 3.0 Relative Gain compared with Random Replay, and 27.96 Relative Gain compared with No Replay.

在本研究中，我们提出了一种称为基于指令的连续学习（InsCL）的新范例，通过使用 Wassertein 距离计算任务相似性并基于此动态地回放先前的数据，实现了基于任务相似性的回放策略，同时引入了指令信息度量（InsInfo）来量化指令的复杂性和多样性，并根据 InsInfo 指导回放过程更倾向于高质量的数据。我们在 16 个不同训练顺序的任务上进行了广泛的实验，观察到 InsCL 持续性能改进。当所有任务都训练完毕时，与随机回放相比，InsCL 实现了 3.0 相对增益，与无回放相比，实现了 27.96 相对增益。

InsCL：一种数据高效的连续学习范式，用于对大型语言模型进行微调并提供指导