Instruction tuning has unlocked powerful capabilities in large language
models (LLMs), effectively using combined datasets to develop generalpurpose
chatbots. However, real-world applications often require a specialized suite of
skills (e.g., reasoning). The challenge lies in identifying the most relevant
data from these extensive datasets to effectively develop specific
capabilities, a setting we frame as targeted instruction tuning. We propose
LESS, an optimizer-aware and practically efficient algorithm to effectively
estimate data influences and perform Low-rank gradiEnt Similarity Search for
instruction data selection. Crucially, LESS adapts existing influence
formulations to work with the Adam optimizer and variable-length instruction
data. LESS first constructs a highly reusable and transferable gradient
datastore with low-dimensional gradient features and then selects examples
based on their similarity to few-shot examples embodying a specific capability.
Experiments show that training on a LESS-selected 5% of the data can often
outperform training on the full dataset across diverse downstream tasks.
Furthermore, the selected data is highly transferable: smaller models can be
leveraged to select useful data for larger models and models from different
families. Our qualitative analysis shows that our method goes beyond surface
form cues to identify data that exemplifies the necessary reasoning skills for
the intended downstream application.

我们提出了 LESS，这是一种优化感知且实际高效的算法，用于有效估计数据影响并执行用于指令数据选择的低秩梯度相似性搜索。实验表明，在多样的下游任务中，使用 LESS 选中的 5% 的数据进行训练通常优于全数据集的训练。此外，所选数据具有高度可迁移性：可以利用较小的模型为较大的模型以及来自不同家族的模型选择有用的数据。我们的定性分析表明，我们的方法超越了表面形式线索，能够识别出在预期的下游应用中体现必要推理技能的数据。