While forward reasoning (i.e. find the answer given the question) has been
explored extensively in the recent literature, backward reasoning is relatively
unexplored. We examine the backward reasoning capabilities of LLMs on Math Word
Problems (MWPs): given a mathematical question and its answer, with some
details omitted from the question, can LLMs effectively retrieve the missing
information?
In this paper, we formally define the backward reasoning task on math word
problems and modify three datasets to evaluate this task: GSM8k, SVAMP and
MultiArith. Our findings show a significant drop in the accuracy of models on
backward reasoning compared to forward reasoning across four SOTA LLMs (GPT4,
GPT3.5, PaLM-2, and LLaMa-2). Utilizing the specific format of this task, we
propose three novel techniques that improve performance: Rephrase reformulates
the given problem into a forward reasoning problem, PAL-Tools combines the idea
of Program-Aided LLMs to produce a set of equations that can be solved by an
external solver, and Check your Work exploits the availability of natural
verifier of high accuracy in the forward direction, interleaving solving and
verification steps. Finally, realizing that each of our base methods correctly
solves a different set of problems, we propose a novel Bayesian formulation for
creating an ensemble over these base methods aided by a verifier to further
boost the accuracy by a significant margin. Extensive experimentation
demonstrates that our techniques successively improve the performance of LLMs
on the backward reasoning task, with the final ensemble-based method resulting
in a substantial performance gain compared to the raw LLMs with standard
prompting techniques such as chain-of-thought.

本文研究倒向推理任务在数学问题中的应用，并提出了三种新技术以及一种基于贝叶斯公式的集成方法，这些方法显著提高了语言模型在倒向推理任务上的性能。

填空题：探索和增强 LLM 在数学应用问题中倒向推理的能力

Fill in the Blank: Exploring and Enhancing LLM Capabilities for Backward  Reasoning in Math Word Problems

Chain-of-Though (CoT) prompting has shown promising performance in various
reasoning tasks. Recently, Self-Consistency \citep{wang2023selfconsistency}
proposes to sample a diverse set of reasoning chains which may lead to
different answers while the answer that receives the most votes is selected. In
this paper, we propose a novel method to use backward reasoning in verifying
candidate answers. We mask a token in the question by ${\bf x}$ and ask the LLM
to predict the masked token when a candidate answer is provided by \textit{a
simple template}, i.e., ``\textit{\textbf{If we know the answer of the above
question is \{a candidate answer\}, what is the value of unknown variable ${\bf
x}$?}}'' Intuitively, the LLM is expected to predict the masked token
successfully if the provided candidate answer is correct. We further propose
FOBAR to combine forward and backward reasoning for estimating the probability
of candidate answers. We conduct extensive experiments on six data sets and
three LLMs. Experimental results demonstrate that FOBAR achieves
state-of-the-art performance on various reasoning benchmarks.

提出了一种结合前向和后向推理的新方法，称为 FOBAR，通过用简单模板生成问题并要求 LLM 预测被掩码的标记来验证候选答案，实验证明 FOBAR 在各种推理基准上达到了最先进的性能。

大型语言模型中的后向推理用于验证

Backward Reasoning in Large Language Models for Verification

Being able to generate informative and coherent dialogue responses is crucial
when designing human-like open-domain dialogue systems. Encoder-decoder-based
dialogue models tend to produce generic and dull responses during the decoding
step because the most predictable response is likely to be a non-informative
response instead of the most suitable one. To alleviate this problem, we
propose to train the generation model in a bidirectional manner by adding a
backward reasoning step to the vanilla encoder-decoder training. The proposed
backward reasoning step pushes the model to produce more informative and
coherent content because the forward generation step's output is used to infer
the dialogue context in the backward direction. The advantage of our method is
that the forward generation and backward reasoning steps are trained
simultaneously through the use of a latent variable to facilitate bidirectional
optimization. Our method can improve response quality without introducing side
information (e.g., a pre-trained topic model). The proposed bidirectional
response generation method achieves state-of-the-art performance for response
quality.

本文提出一种双向推理的生成模型，通过向基础的编码解码模型添加后向推理步骤，推动模型产生更多信息丰富、连贯的回复，实现了双向优化，最终提升了响应质量，成为当前响应质量最好的方法之一。