Learning paradigms for large language models (LLMs) currently tend to fall
within either in-context learning (ICL) or full fine-tuning. Each of these
comes with their own trade-offs based on available data, model size, compute
cost, ease-of-use, and final quality with neither solution performing well
across-the-board. In this article, we first describe ICL and fine-tuning
paradigms in a way that highlights their natural connections. Based on these
connections, we propose a new learning paradigm called FIAT that fuses the best
of these paradigms together, enabling prompt-engineered instructions and
chain-of-thought reasoning with the very largest models while also using
similar methods to perform parameter updates on a modestly-sized LLM with
parameter-efficient tuning. We evaluate FIAT's effectiveness on a variety of
multilingual tasks and observe that FIAT performs better than both ICL and
fine-tuning at scales ranging from 100-10,000 training examples. We hope that
FIAT provides a practical way of harnessing the full potential of LLMs without
needing to make a hard choice between learning paradigms.

使用两种学习模式，即上下文学习（ICL）和完全微调，以及一种新的学习范式 FIAT，该范式将二者结合起来，使得在使用最大尺寸模型的同时，可以进行快速引擎指令和思维链推理，并利用类似方法对规模适度的 LLM 进行参数更新。我们发现，FIAT 在各种多语言任务上的表现优于 ICL 和完全微调，训练样本范围为 100-10,000。希望 FIAT 为充分发挥 LLMs 的潜力提供了一种实用的方式，无需在学习范式之间进行困难选择。