Chain-of-thought prompting (CoT) advances the reasoning abilities of large
language models (LLMs) and achieves superior performance in arithmetic,
commonsense, and symbolic reasoning tasks. However, most CoT studies rely on
carefully designed human-annotated rational chains to prompt the language
model, which poses challenges for real-world applications where labeled
training data is available without human-annotated rational chains. This
creates barriers to applications of CoT prompting to these general tasks. This
paper proposes a new strategy, Automate-CoT (Automatic Prompt Augmentation and
Selection with Chain-of-Thought), that can bypass human engineering of CoTs by
automatically augmenting rational chains from a small labeled dataset, and then
pruning low-quality chains to construct a candidate pool of machine-generated
rationale chains based on the labels. Finally, it selects the optimal
combination of several rationale chains from the pool for CoT prompting by
employing a variance-reduced policy gradient strategy to estimate the
significance of each example in a black-box language model. Automate-CoT
enables a quick adaptation of the CoT technique to different tasks.
Experimental results demonstrate the effectiveness of our method, where
state-of-the-art results are achieved on arithmetic reasoning (+2.7\%),
commonsense reasoning (+3.4\%), symbolic reasoning (+3.2\%), and non-reasoning
tasks (+2.5\%). Our code will be available at
this https URL.

本文提出了一种使用 Automate-CoT 自动扩展和选择合理链来推动 CoT 的方法，该方法通过构建候选池从标签生成的机器理性链中选择多个理性链的最佳组合，以推进大型语言模型的推理能力，并在算术推理、常识推理、符号推理和非推理任务中取得了最先进的结果。