Spurred by advancements in scale, large language models (LLMs) have
demonstrated strong few-shot learning ability via in-context learning (ICL).
However, the performance of ICL has been shown to be highly sensitive to the
selection of few-shot demonstrations. Selecting the most suitable examples as
context remains an ongoing challenge and an open problem. Existing literature
has highlighted the importance of selecting examples that are diverse or
semantically similar to the test sample while ignoring the fact that the
optimal selection dimension, i.e., diversity or similarity, is task-specific.
Leveraging the merits of both dimensions, we propose Iterative Demonstration
Selection (IDS). Using zero-shot chain-of-thought reasoning (Zero-shot-CoT),
IDS iteratively selects examples that are diverse but still strongly correlated
with the test sample as ICL demonstrations. Specifically, IDS applies
Zero-shot-CoT to the test sample before demonstration selection. The output
reasoning path is then used to choose demonstrations that are prepended to the
test sample for inference. The generated answer is accompanied by its
corresponding reasoning path for extracting a new set of demonstrations in the
next iteration. After several iterations, IDS adopts majority voting to obtain
the final result. Through extensive experiments on tasks including commonsense
reasoning, question answering, topic classification, and sentiment analysis, we
demonstrate that IDS can consistently outperform existing ICL demonstration
selection methods.

基于大规模语言模型的研究表明，通过上下文学习（ICL），在选择示范的少量样例时，可实现强有力的少样本学习能力。然而，ICL 的性能对少样本示范的选择非常敏感，因此选择最适合的上下文示范仍然是一个持续的挑战和一个开放性问题。本研究提出了迭代示范选择（IDS）方法，通过利用零样本思维推理（Zero-shot-CoT），IDS 迭代地选择既具有多样性又与测试样本强相关的示范。通过在示范选择之前对测试样本应用零样本思维推理，IDS 选择思维路径来作为推理的示范，从而生成答案并得到下一次迭代中用于提取新示范的相关推理路径。经过多次迭代后，IDS 采用多数投票的方法获得最终结果。通过在常识推理、问答、主题分类和情感分析等任务上进行了大量实验证明，IDS 能够一直优于现有的 ICL 示范选择方法。