Prompting Large Language Models (LLMs), or providing context on the expected model of operation, is an effective way to steer the outputs of such models to satisfy human desiderata after they have been trained. But in rapidly evolving domains, there is often need to fine-tune LLMs to improve either the kind of knowledge in their memory or their abilities to perform open ended reasoning in new domains. When human's learn new concepts, we often do so by linking the new material that we are studying to concepts we have already learned before. To that end, we ask, "can prompting help us teach LLMs how to learn". In this work, we study a novel generalization of instruction tuning, called contextual fine-tuning, to fine-tune LLMs. Our method leverages instructional prompts designed to mimic human cognitive strategies in learning and problem-solving to guide the learning process during training, aiming to improve the model's interpretation and understanding of domain-specific knowledge. We empirically demonstrate that this simple yet effective modification improves the ability of LLMs to be fine-tuned rapidly on new datasets both within the medical and financial domains.

本研究解决了大型语言模型（LLMs）在迅速变化领域中知识更新和新领域推理能力不足的问题。提出了一种新的上下文微调方法，利用模拟人类认知策略的指令提示来引导模型学习过程，从而增强模型对领域特定知识的理解。实验证明，这一简单有效的改进显著提升了LLMs在医学和金融领域快速微调新数据集的能力。

通过上下文微调教会大型语言模型如何学习