This paper studies the relationship between the surface form of a
mathematical problem and its solvability by large language models. We find that
subtle alterations in the surface form can significantly impact the answer
distribution and the solve rate, exposing the language model's lack of
robustness and sensitivity to the surface form in reasoning through complex
problems. To improve mathematical reasoning performance, we propose
Self-Consistency-over-Paraphrases (SCoP), which diversifies reasoning paths
from specific surface forms of the problem. We evaluate our approach on four
mathematics reasoning benchmarks over three large language models and show that
SCoP improves mathematical reasoning performance over vanilla self-consistency,
particularly for problems initially deemed unsolvable. Finally, we provide
additional experiments and discussion regarding problem difficulty and surface
forms, including cross-model difficulty agreement and paraphrasing
transferability, and Variance of Variations (VOV) for language model
evaluation.

该研究探讨了数学问题的表面形式与其是否能被大型语言模型解决之间的关系，并发现表面形式的微小改变可以显著影响答案分布和解决率，揭示了语言模型在推理复杂问题时缺乏鲁棒性和对表面形式的敏感性。为了提高数学推理性能，研究提出了自洽性超重述（SCoP）方法，该方法通过多样化问题的特定表面形式来多样化推理路径。通过对三个大型语言模型上的四个数学推理基准进行评估，研究表明 SCoP 相对于普通的自洽性方法可以提高数学推理性能，特别是对于最初被认为无法解决的问题。最后，研究提供了关于问题难度和表面形式的额外实验和讨论，包括模型间的难度一致性和重述的可迁移性，以及用于语言模型评估的变异性。

剽窃和解决：探索并利用表面形式对大型语言模型的数学推理影响

Paraphrase and Solve: Exploring and Exploiting the Impact of Surface  Form on Mathematical Reasoning in Large Language Models

Generating from Abstract Meaning Representation (AMR) is an underspecified
problem, as many syntactic decisions are not constrained by the semantic graph.
To explicitly account for this underspecification, we break down generating
from AMR into two steps: first generate a syntactic structure, and then
generate the surface form. We show that decomposing the generation process this
way leads to state-of-the-art single model performance generating from AMR
without additional unlabelled data. We also demonstrate that we can generate
meaning-preserving syntactic paraphrases of the same AMR graph, as judged by
humans.

将从 “抽象意义表达”（AMR）生成的过程分解成两个步骤，即首先生成句法结构，然后生成表面形式，从而实现了最先进的单模型性能，同时生成了与原 AMR 图形意义相同的句法释义。