Fine-tuning pretrained language models has shown promising results on a wide
range of tasks, but when encountering a novel task, do they rely more on
generic pretrained representation, or develop brand new task-specific
solutions? Here, we fine-tuned GPT-2 on a context-dependent decision-making
task, novel to the model but adapted from neuroscience literature. We compared
its performance and internal mechanisms to a version of GPT-2 trained from
scratch on the same task. Our results show that fine-tuned models depend
heavily on pretrained representations, particularly in later layers, while
models trained from scratch develop different, more task-specific mechanisms.
These findings highlight the advantages and limitations of pretraining for task
generalization and underscore the need for further investigation into the
mechanisms underpinning task-specific fine-tuning in LLMs.

微调预训练语言模型在广泛的任务上展现出有希望的结果，但面对新任务时，它们更依赖于通用的预训练表示还是开发全新的任务特定解决方案？我们在一个与模型相关且来自神经科学文献的情境依赖决策任务上微调了 GPT-2，并将其性能和内部机制与从头开始训练的 GPT-2 版本进行了比较。我们的结果显示，微调模型在较后层次上严重依赖于预训练表示，而从头开始训练的模型则开发了不同、更具任务特定性的机制。这些发现突出了预训练在任务泛化中的优势和限制，并强调了进一步研究语言模型微调的任务特定机制的必要性。