Large language models have recently shown promising progress in mathematical
reasoning when fine-tuned with human-generated sequences walking through a
sequence of solution steps. However, the solution sequences are not formally
structured and the resulting model-generated sequences may not reflect the kind
of systematic reasoning we might expect an expert human to produce. In this
paper, we study how to build stronger reasoning capability in language models
using the idea of relational abstractions. We introduce new types of sequences
that more explicitly provide an abstract characterization of the transitions
through intermediate solution steps to the goal state. We find that models that
are supplied with such sequences as prompts can solve tasks with a
significantly higher accuracy, and models that are trained to produce such
sequences solve problems better than those that are trained with previously
used human-generated sequences and other baselines. Our work thus takes several
steps toward elucidating and improving how language models perform on tasks
requiring multi-step mathematical reasoning.

本文介绍了一种利用关系抽象概念来帮助大型语言模型在数学推理方面提高的新方法，并发现将这种序列用作提示可使模型具有更强的推理能力，特别是在需要执行多步任务的情况下。

利用关系抽象进行推理的学习

Learning to Reason With Relational Abstractions

Reinforcement learning in problems with symbolic state spaces is challenging
due to the need for reasoning over long horizons. This paper presents a new
approach that utilizes relational abstractions in conjunction with deep
learning to learn a generalizable Q-function for such problems. The learned
Q-function can be efficiently transferred to related problems that have
different object names and object quantities, and thus, entirely different
state spaces. We show that the learned generalized Q-function can be utilized
for zero-shot transfer to related problems without an explicit, hand-coded
curriculum. Empirical evaluations on a range of problems show that our method
facilitates efficient zero-shot transfer of learned knowledge to much larger
problem instances containing many objects.

本文介绍了一种新的强化学习方法，利用关系抽象和深度学习，学习符号状态空间中可推广的 Q 函数，并能在零阶转移下转移到不同的相关问题，实现对大规模问题的有效知识迁移。