Automated math word problem solvers based on neural networks have
successfully managed to obtain 70-80\% accuracy in solving arithmetic word
problems. However, it has been shown that these solvers may rely on superficial
patterns to obtain their equations. In order to determine what information math
word problem solvers use to generate solutions, we remove parts of the input
and measure the model's performance on the perturbed dataset. Our results show
that the model is not sensitive to the removal of many words from the input and
can still manage to find a correct answer when given a nonsense question. This
indicates that automatic solvers do not follow the semantic logic of math word
problems, and may be overfitting to the presence of specific words.

基于神经网络的自动数学问题求解器在解决算术问题方面成功地达到了 70-80％的准确率，然而研究表明这些求解器可能依赖表面模式得到方程。为了确定数学问题求解器使用哪些信息生成解答，我们移除输入的一部分并测量模型对扰动数据集的表现。结果表明，当给出无意义问题时，即使从输入中删除许多单词，模型也不敏感并仍能找到正确答案。这表明自动求解器并不遵循数学问题的语义逻辑，可能过拟合于特定单词的存在。

数学单词问题解题器的解释

Explaining Math Word Problem Solvers

Symbolic regression (SR) is a challenging task in machine learning that
involves finding a mathematical expression for a function based on its values.
Recent advancements in SR have demonstrated the efficacy of pretrained
transformer-based models for generating equations as sequences, which benefit
from large-scale pretraining on synthetic datasets and offer considerable
advantages over GP-based methods in terms of inference time. However, these
models focus on supervised pretraining goals borrowed from text generation and
ignore equation-specific objectives like accuracy and complexity. To address
this, we propose TPSR, a Transformer-based Planning strategy for Symbolic
Regression that incorporates Monte Carlo Tree Search into the transformer
decoding process. TPSR, as opposed to conventional decoding strategies, allows
for the integration of non-differentiable feedback, such as fitting accuracy
and complexity, as external sources of knowledge into the equation generation
process. Extensive experiments on various datasets show that our approach
outperforms state-of-the-art methods, enhancing the model's fitting-complexity
trade-off, extrapolation abilities, and robustness to noise. We also
demonstrate that the utilization of various caching mechanisms can further
enhance the efficiency of TPSR.

提出了一种基于变压器（Transformer）的符号回归规划策略（TPSR），它融合了蒙特卡罗树搜索，允许将准确性和复杂性等非可微反馈信息整合到方程生成过程中，并且在多个数据集上进行的广泛实验表明，TPSR 方法优于现有的方法。