Recent advancements in Chain-of-Thought prompting have facilitated
significant breakthroughs for Large Language Models (LLMs) in complex reasoning
tasks. Current research enhances the reasoning performance of LLMs by sampling
multiple reasoning chains and ensembling based on the answer frequency.
However, this approach fails in scenarios where the correct answers are in the
minority. We identify this as a primary factor constraining the reasoning
capabilities of LLMs, a limitation that cannot be resolved solely based on the
predicted answers. To address this shortcoming, we introduce a hierarchical
reasoning aggregation framework AoR (Aggregation of Reasoning), which selects
answers based on the evaluation of reasoning chains. Additionally, AoR
incorporates dynamic sampling, adjusting the number of reasoning chains in
accordance with the complexity of the task. Experimental results on a series of
complex reasoning tasks show that AoR outperforms prominent ensemble methods.
Further analysis reveals that AoR not only adapts various LLMs but also
achieves a superior performance ceiling when compared to current methods.

最近在 “思维链路提示” 方面取得的进展为大型语言模型在复杂推理任务中带来了重大突破。然而，当前的研究通过对答案频率进行多个推理链路的抽样和集成来增强 LLMs 的推理性能，但这种方法在正确答案占少数的情况下会失败。为了解决这个局限性，我们提出了一种层次化推理聚合框架 AoR（聚合推理），它根据推理链路的评估选择答案，并结合动态抽样，根据任务的复杂性调整推理链路的数量。对一系列复杂推理任务的实验结果表明，AoR 优于其他突出的集成方法。进一步分析表明，与当前方法相比，AoR 不仅适应多种 LLMs，而且达到了更高的性能上限。