Large Language Models (LLMs) have achieved remarkable success, demonstrating
powerful instruction-following capabilities across diverse tasks. Instruction
fine-tuning is critical in enabling LLMs to align with user intentions and
effectively follow instructions. In this work, we investigate how instruction
fine-tuning modifies pre-trained models, focusing on two perspectives:
instruction recognition and knowledge evolution. To study the behavior shift of
LLMs, we employ a suite of local and global explanation methods, including a
gradient-based approach for input-output attribution and techniques for
interpreting patterns and concepts in self-attention and feed-forward layers.
Our findings reveal three significant impacts of instruction fine-tuning: 1) It
empowers LLMs to better recognize the instruction parts from user prompts,
thereby facilitating high-quality response generation and addressing the
``lost-in-the-middle'' issue observed in pre-trained models; 2) It aligns the
knowledge stored in feed-forward layers with user-oriented tasks, exhibiting
minimal shifts across linguistic levels. 3) It facilitates the learning of
word-word relations with instruction verbs through the self-attention
mechanism, particularly in the lower and middle layers, indicating enhanced
recognition of instruction words. These insights contribute to a deeper
understanding of the behavior shifts in LLMs after instruction fine-tuning and
lay the groundwork for future research aimed at interpreting and optimizing
LLMs for various applications. We will release our code and data soon.

通过本研究，我们发现指导微调对大型语言模型产生了三个重要影响，包括了对指令识别的加强、对知识存储层次的对齐以及对单词关系学习的促进。这些发现有助于更深入地理解指导微调对大型语言模型行为变化的影响，并为未来解释和优化这些模型以适用于不同应用领域的研究打下了基础。

从语言建模到指令遵循：理解指令调优后 LLMs 的行为转变

From Language Modeling to Instruction Following: Understanding the  Behavior Shift in LLMs after Instruction Tuning

Deep learning relies on the availability of a large corpus of data (labeled
or unlabeled). Thus, one challenging unsettled question is: how to train a deep
network on a relatively small dataset? To tackle this question, we propose an
evolution-inspired training approach to boost performance on relatively small
datasets. The knowledge evolution (KE) approach splits a deep network into two
hypotheses: the fit-hypothesis and the reset-hypothesis. We iteratively evolve
the knowledge inside the fit-hypothesis by perturbing the reset-hypothesis for
multiple generations. This approach not only boosts performance, but also
learns a slim network with a smaller inference cost. KE integrates seamlessly
with both vanilla and residual convolutional networks. KE reduces both
overfitting and the burden for data collection.
We evaluate KE on various network architectures and loss functions. We
evaluate KE using relatively small datasets (e.g., CUB-200) and randomly
initialized deep networks. KE achieves an absolute 21% improvement margin on a
state-of-the-art baseline. This performance improvement is accompanied by a
relative 73% reduction in inference cost. KE achieves state-of-the-art results
on classification and metric learning benchmarks. Code available at
this http URL

提出了一种基于知识进化的训练方法，它通过将深度网络分成适配假设和重置假设来迭代进化知识，能够在相对较小的数据集上提高性能、学习较瘦的网络并减少推理成本，达到 21% 的绝对性能提升和 73% 的相对推理成本降低。