Even though large language models (LLMs) have demonstrated remarkable
capability in solving various natural language tasks, the capability of an LLM
to follow human instructions is still a concern. Recent works have shown great
improvements in the instruction-following capability via additional training
for instruction-following tasks. However, the mechanisms responsible for
effective instruction-following capabilities remain inadequately understood.
Here, we introduce a simplified instruction-following task and use synthetic
datasets to analyze a Transformer-based causal language model. Our findings
suggest that the model learns task-specific information by clustering data
within its hidden space, with this clustering process evolving dynamically
during learning. We also demonstrate how this phenomenon assists the model in
handling unseen instances and validate our results in a more realistic setting.

通过对合成数据集的分析，我们发现大规模语言模型通过将数据在其隐藏空间内进行聚类学习特定任务的信息，并且这个聚类过程在学习过程中动态演化，从而帮助模型处理未知的实例。