Recent work has addressed textual reasoning tasks by prompting large language
models with explanations via the chain-of-thought paradigm. However, subtly
different explanations can yield widely varying downstream task accuracy, so
explanations that have not been "tuned" for a task, such as off-the-shelf
explanations written by nonexperts, may lead to mediocre performance. This
paper tackles the problem of how to optimize explanation-infused prompts in a
black-box fashion. We first generate sets of candidate explanations for each
example in the prompt using a leave-one-out scheme. We then use a two-stage
framework where we first evaluate explanations for each in-context example in
isolation according to proxy metrics. Finally, we search over sets of
explanations to find a set which yields high performance against a
silver-labeled development set, drawing inspiration from recent work on
bootstrapping language models on unlabeled data. Across four textual reasoning
tasks spanning question answering, mathematical reasoning, and natural language
inference, results show that our proxy metrics correlate with ground truth
accuracy and our overall method can effectively improve prompts over
crowdworker annotations and naive search strategies.

本文提出了一种黑盒优化解释注入的提示方法，以提高文本推理任务的性能，在问题回答、数学推理和自然语言推理等四个文本推理任务中实现了有效的促进效果。