Instruction tuning (IT) achieves impressive zero-shot generalization results
by training large language models (LLMs) on a massive amount of diverse tasks
with instructions. However, how to select new tasks to improve the performance
and generalizability of IT models remains an open question. Training on all
existing tasks is impractical due to prohibiting computation requirements, and
randomly selecting tasks can lead to suboptimal performance. In this work, we
propose active instruction tuning based on prompt uncertainty, a novel
framework to identify informative tasks, and then actively tune the models on
the selected tasks. We represent the informativeness of new tasks with the
disagreement of the current model outputs over perturbed prompts. Our
experiments on NIV2 and Self-Instruct datasets demonstrate that our method
consistently outperforms other baseline strategies for task selection,
achieving better out-of-distribution generalization with fewer training tasks.
Additionally, we introduce a task map that categorizes and diagnoses tasks
based on prompt uncertainty and prediction probability. We discover that
training on ambiguous (prompt-uncertain) tasks improves generalization while
training on difficult (prompt-certain and low-probability) tasks offers no
benefit, underscoring the importance of task selection for instruction tuning.

通过基于提示不确定性的主动指导调整，本研究提出了一种新颖的框架来识别信息量大的任务，并在所选任务上主动调整模型，实验表明该方法在任务选择方面优于其他基准策略，并通过更少的训练任务实现了更好的超出分布的泛化性能。