Large language models (LLMs) have been explored in a variety of reasoning
tasks including solving of mathematical problems. Each math dataset typically
includes its own specially designed evaluation script, which, while suitable
for its intended use, lacks generalizability across different datasets.
Consequently, updates and adaptations to these evaluation tools tend to occur
without being systematically reported, leading to inconsistencies and obstacles
to fair comparison across studies. To bridge this gap, we introduce a
comprehensive mathematical evaluation toolkit that not only utilizes a python
computer algebra system (CAS) for its numerical accuracy, but also integrates
an optional LLM, known for its considerable natural language processing
capabilities. To validate the effectiveness of our toolkit, we manually
annotated two distinct datasets. Our experiments demonstrate that the toolkit
yields more robust evaluation results compared to prior works, even without an
LLM. Furthermore, when an LLM is incorporated, there is a notable enhancement.
The code for our method will be made available at
https://github.com/MARIO-Math-Reasoning/math_evaluation.

大语言模型（LLMs）在解决数学问题等各种推理任务中得到了探索。我们引入了一个全面的数学评估工具包，不仅利用 python 计算代数系统（CAS）进行数值精度评估，还集成了一个可选的大语言模型（LLM），以验证工具包的有效性。