Despite the surprising few-shot performance of in-context learning (ICL), it
is still a common practice to randomly sample examples to serve as context.
This paper advocates a new principle for ICL: self-adaptive in-context
learning. The self-adaption mechanism is introduced to help each sample find an
in-context example permutation (i.e., selection and ordering) that can derive
the correct prediction, thus maximizing performance. To validate the
effectiveness of self-adaptive ICL, we propose a general select-then-rank
framework and instantiate it with new selection and ranking algorithms. Upon
extensive evaluation on eight different NLP datasets, our self-adaptive ICL
method achieves a 40% relative improvement over the common practice setting.
Further analysis reveals the enormous potential of self-adaptive ICL that it
might be able to close the gap between ICL and finetuning given more advanced
algorithms. Our code is released to facilitate future research in this area:
this https URL

本文提出了一种基于自适应机制的 in-context learning (ICL) 原则，通过选择和调整上下文示例来实现正确预测，从而最大化性能，并在八个不同 NLP 数据集上进行了全面评估，相对于常见实践设置，我们的 ICL 方法实现了 40% 的性能改进。