Large language models (LLMs) have the ability to perform in-context learning
(ICL) of new tasks by conditioning on prompts comprising a few task examples.
This work studies the problem of selecting the best examples given a candidate
pool to improve ICL performance on given a test input. Existing approaches
either require training with feedback from a much larger LLM or are
computationally expensive. We propose a novel metric, GistScore, based on
Example Gisting, a novel approach for training example retrievers for ICL using
an attention bottleneck via Gisting, a recent technique for compressing task
instructions. To tradeoff performance with ease of use, we experiment with both
fine-tuning gist models on each dataset and multi-task training a single model
on a large collection of datasets. On 21 diverse datasets spanning 9 tasks, we
show that our fine-tuned models get state-of-the-art ICL performance with 20%
absolute average gain over off-the-shelf retrievers and 7% over the best prior
methods. Our multi-task model generalizes well out-of-the-box to new task
categories, datasets, and prompt templates with retrieval speeds that are
consistently thousands of times faster than the best prior training-free
method.

使用 GistScore 度量示例 Gisting 的新方法，基于 Fine-tuned 模型实现最新的 in-context learning（ICL），在 21 个不同数据集上，可以获得超过 20% 绝对平均增益，且速度比之前最佳的非训练法提高了数千倍。