Numeral systems and units of measurement are two conjoined topics in
activities of human beings and have mutual effects with the languages
expressing them. Currently, the evaluation of Large Language Models (LLMs)
often involves mathematical reasoning, yet little attention is given to how
minor changes in numbers or units can drastically alter the complexity of
problems and the performance of LLMs. In this paper, we scrutinize existing
LLMs on processing of numerals and units of measurement by constructing
datasets with perturbations. We first anatomize the reasoning of math word
problems to different sub-procedures like numeral conversions from language to
numbers and measurement conversions based on units. Then we further annotate
math word problems from ancient Chinese arithmetic works which are challenging
in numerals and units of measurement. Experiments on perturbed datasets
demonstrate that LLMs still encounter difficulties in handling numeral and
measurement conversions.

人类的数字系统和度量单位是两个相互关联的主题，它们与表达这些主题的语言有着相互影响。本文通过构建带有扰动的数据集，研究已有的大型语言模型在处理数字和度量单位时遇到的困难，从语言到数字的数字转换和基于单位的度量转换都是数学问题的不同子过程。实验表明，大型语言模型在处理数字和度量转换方面仍然存在困难。