Many recent developments in large language models focus on prompting them to
perform specific tasks. One effective prompting method is in-context learning,
where the model performs a (possibly new) generation/prediction task given one
(or more) examples. Past work has shown that the choice of examples can make a
large impact on task performance. However, finding good examples is not
straightforward since the definition of a representative group of examples can
vary greatly depending on the task. While there are many existing methods for
selecting in-context examples, they generally score examples independently,
ignoring the dependency between them and the order in which they are provided
to the large language model. In this work, we propose Retrieval for In-Context
Learning (RetICL), a learnable method for modeling and optimally selecting
examples sequentially for in-context learning. We frame the problem of
sequential example selection as a Markov decision process, design an example
retriever model using an LSTM, and train it using proximal policy optimization
(PPO). We validate RetICL on math problem solving datasets and show that it
outperforms both heuristic and learnable baselines, and achieves
state-of-the-art accuracy on the TabMWP dataset. We also use case studies to
show that RetICL implicitly learns representations of math problem solving
strategies.

本研究提出了 Retrieval for In-Context Learning (RetICL)，一种可学习的方法，用于模拟和最佳选择逐个该如何为 in-context learning 选择任务例子。它将顺序示例选择问题作为马尔可夫决策过程，使用 LSTM 设计示例检索器模型，并使用 PPO 进行训练。我们在数学问题求解数据集上验证了 RetICL，表明它优于启发式和可学习的基线，并在 TabMWP 数据集上实现了最先进的准确性。我们还使用案例研究展示了 RetICL 隐含学习了数学问题求解策略的表示方式。