We introduce iterative retrieval, a novel framework that empowers retrievers
to make iterative decisions through policy optimization. Finding an optimal
portfolio of retrieved items is a combinatorial optimization problem, generally
considered NP-hard. This approach provides a learned approximation to such a
solution, meeting specific task requirements under a given family of large
language models (LLMs). We propose a training procedure based on reinforcement
learning, incorporating feedback from LLMs. We instantiate an iterative
retriever for composing in-context learning (ICL) exemplars and apply it to
various semantic parsing tasks that demand synthesized programs as outputs. By
adding only 4M additional parameters for state encoding, we convert an
off-the-shelf dense retriever into a stateful iterative retriever,
outperforming previous methods in selecting ICL exemplars on semantic parsing
datasets such as CalFlow, TreeDST, and MTOP. Additionally, the trained
iterative retriever generalizes across different inference LLMs beyond the one
used during training.

通过强化学习的训练过程，我们引入了迭代检索的新框架，将活跃检索器赋予迭代决策的能力，并在选择上下文学习示例时在语义解析任务中表现优于之前的方法。