Recent advances in natural language processing, primarily propelled by Large
Language Models (LLMs), have showcased their remarkable capabilities grounded
in in-context learning. A promising avenue for guiding LLMs in intricate
reasoning tasks involves the utilization of intermediate reasoning steps within
the Chain-of-Thought (CoT) paradigm. Nevertheless, the central challenge lies
in the effective selection of exemplars for facilitating in-context learning.
In this study, we introduce a framework that leverages Dual Queries and
Low-rank approximation Re-ranking (DQ-LoRe) to automatically select exemplars
for in-context learning. Dual Queries first query LLM to obtain LLM-generated
knowledge such as CoT, then query the retriever to obtain the final exemplars
via both question and the knowledge. Moreover, for the second query, LoRe
employs dimensionality reduction techniques to refine exemplar selection,
ensuring close alignment with the input question's knowledge. Through extensive
experiments, we demonstrate that DQ-LoRe significantly outperforms prior
state-of-the-art methods in the automatic selection of exemplars for GPT-4,
enhancing performance from 92.5\% to 94.2\%. Our comprehensive analysis further
reveals that DQ-LoRe consistently outperforms retrieval-based approaches in
terms of both performance and adaptability, especially in scenarios
characterized by distribution shifts. DQ-LoRe pushes the boundaries of
in-context learning and opens up new avenues for addressing complex reasoning
challenges. We will release the code soon.

通过利用双重查询和低秩逼近重排 (DQ-LoRe) 框架自动选择实例来进行上下文学习，我们展示了 DQ-LoRe 明显优于先前最先进方法，在 GPT-4 的实例自动选择中将性能从 92.5% 提升到 94.2%。我们的综合分析进一步揭示了 DQ-LoRe 在性能和适应性方面始终优于基于检索的方法，特别是在分布转移情景下。 DQ-LoRe 推动了上下文学习的界限，为解决复杂的推理挑战开辟了新的途径。

DQ-LoRe: 双查询经过低秩近似重排序进行上下文学习

DQ-LoRe: Dual Queries with Low Rank Approximation Re-ranking for  In-Context Learning

Recent work has addressed textual reasoning tasks by prompting large language
models with explanations via the chain-of-thought paradigm. However, subtly
different explanations can yield widely varying downstream task accuracy, so
explanations that have not been "tuned" for a task, such as off-the-shelf
explanations written by nonexperts, may lead to mediocre performance. This
paper tackles the problem of how to optimize explanation-infused prompts in a
black-box fashion. We first generate sets of candidate explanations for each
example in the prompt using a leave-one-out scheme. We then use a two-stage
framework where we first evaluate explanations for each in-context example in
isolation according to proxy metrics. Finally, we search over sets of
explanations to find a set which yields high performance against a
silver-labeled development set, drawing inspiration from recent work on
bootstrapping language models on unlabeled data. Across four textual reasoning
tasks spanning question answering, mathematical reasoning, and natural language
inference, results show that our proxy metrics correlate with ground truth
accuracy and our overall method can effectively improve prompts over
crowdworker annotations and naive search strategies.

本文提出了一种黑盒优化解释注入的提示方法，以提高文本推理任务的性能，在问题回答、数学推理和自然语言推理等四个文本推理任务中实现了有效的促进效果。