Large language models have recently shown promising progress in mathematical
reasoning when fine-tuned with human-generated sequences walking through a
sequence of solution steps. However, the solution sequences are not formally
structured and the resulting model-generated sequences may not reflect the kind
of systematic reasoning we might expect an expert human to produce. In this
paper, we study how to build stronger reasoning capability in language models
using the idea of relational abstractions. We introduce new types of sequences
that more explicitly provide an abstract characterization of the transitions
through intermediate solution steps to the goal state. We find that models that
are supplied with such sequences as prompts can solve tasks with a
significantly higher accuracy, and models that are trained to produce such
sequences solve problems better than those that are trained with previously
used human-generated sequences and other baselines. Our work thus takes several
steps toward elucidating and improving how language models perform on tasks
requiring multi-step mathematical reasoning.

本文介绍了一种利用关系抽象概念来帮助大型语言模型在数学推理方面提高的新方法，并发现将这种序列用作提示可使模型具有更强的推理能力，特别是在需要执行多步任务的情况下。