Numerical reasoning over natural language has been a long-standing goal for
the research community. However, cutting-edge language models have proven
difficult to reliably generalize to a broad range of numbers, although they
have shown proficiency in reasoning over common and simple numbers. In this
paper, we propose a novel method to elicit and exploit the numerical reasoning
knowledge hidden in pre-trained language models using simple anchor numbers.
Concretely, we first leverage simple numbers as anchors to probe the implicitly
inferred arithmetic expressions from language models, and then explicitly apply
the expressions on complex numbers to get corresponding answers. To inversely
elicit arithmetic expressions, we transform and formulate the task as an
analytically solvable linear system. Experimental results on several numerical
reasoning benchmarks demonstrate that our approach significantly improves
numerical reasoning capabilities of existing LMs. More importantly, our
approach is training-free and simply works in the inference phase, making it
highly portable and achieving consistent performance benefits across a variety
of language models (GPT-3, T5, BART, etc) in all zero-shot, few-shot, and
fine-tuning scenarios.

本文提出了一种利用锚定数字来诱发和利用预先训练的语言模型潜在的数字推理知识的新方法，通过在复杂的数字上应用简单的锚定数字隐含的推理表达式，以显式地得到相应的答案并训练语言模型的数字推理能力。实验结果表明该方法显著提高了现有语言模型的数字推理能力。