Current representations used in reasoning steps of large language models can
mostly be categorized into two main types: (1) natural language, which is
difficult to verify; and (2) non-natural language, usually programming code,
which is difficult for people who are unfamiliar with coding to read. In this
paper, we propose to use a semi-structured form to represent reasoning steps of
large language models. Specifically, we use relation tuples, which are not only
human-readable but also machine-friendly and easier to verify than natural
language. We implement a framework that includes three main components: (1)
introducing relation tuples into the reasoning steps of large language models;
(2) implementing an automatic verification process of reasoning steps with a
local code interpreter based on relation tuples; and (3) integrating a simple
and effective dynamic feedback mechanism, which we found helpful for
self-improvement of large language models. The experimental results on various
arithmetic datasets demonstrate the effectiveness of our method in improving
the arithmetic reasoning ability of large language models. The source code is
available at this https URL

使用关系元组在大型语言模型的推理步骤中表示一种半结构化形式，通过引入关系元组、自动验证过程和动态反馈机制来改善大型语言模型的算数推理能力。

通过关系元组、验证和动态反馈提高大型语言模型的算术推理能力

Improving Arithmetic Reasoning Ability of Large Language Models through  Relation Tuples, Verification and Dynamic Feedback

A relation tuple consists of two entities and the relation between them, and
often such tuples are found in unstructured text. There may be multiple
relation tuples present in a text and they may share one or both entities among
them. Extracting such relation tuples from a sentence is a difficult task and
sharing of entities or overlapping entities among the tuples makes it more
challenging. Most prior work adopted a pipeline approach where entities were
identified first followed by finding the relations among them, thus missing the
interaction among the relation tuples in a sentence. In this paper, we propose
two approaches to use encoder-decoder architecture for jointly extracting
entities and relations. In the first approach, we propose a representation
scheme for relation tuples which enables the decoder to generate one word at a
time like machine translation models and still finds all the tuples present in
a sentence with full entity names of different length and with overlapping
entities. Next, we propose a pointer network-based decoding approach where an
entire tuple is generated at every time step. Experiments on the publicly
available New York Times corpus show that our proposed approaches outperform
previous work and achieve significantly higher F1 scores.

本文提出了两种使用编码器 - 解码器结构的方法来共同提取关系元组和实体，其中第一种方法采用关系元组表示方案，使解码器能够像机器翻译模型一样逐个单词生成，并仍然找到具有不同长度和重叠实体的所有元组，而第二种方法采用指针网络解码方法，每个时间步长生成整个元组。实验结果表明，本文提出的方法优于先前的工作，并实现了显着更高的 F1 分数。